Skip to content

Latest commit

 

History

History
70 lines (55 loc) · 2.99 KB

README.md

File metadata and controls

70 lines (55 loc) · 2.99 KB
Raw

CounterfactualRegret.jl

codecov

Installation

using Pkg; Pkg.add("CounterfactualRegret")

Implemented Solvers

Solver Name   Description        
CFRSolver   Vanilla CFR solver
CSCFRSolver Chance Sampling    
ESCFRSolver External Sampling  
OSCFRSolver Outcome Sampling  

Each solver takes optional kwarg method, which can be an instantiation of either Vanilla, Plus, or Discount types, which correspond to Vanilla CFR, CFR+, and discounted CFR respectively.

External Solver Packages

Name Status
DeepCFR CI codecov
ESCHER CI codecov
POMDPBestResponse CI codecov

Solving a Matrix Game

using CounterfactualRegret
const CFR = CounterfactualRegret
using CounterfactualRegret.Games
using Plots

game = MatrixGame([
    (1,1) (0,0) (0,0);
    (0,0) (0,2) (3,0);
    (0,0) (2,0) (0,3);
])
sol = CFRSolver(game; debug=true)
cb = CFR.ExploitabilityCallback(sol, 10) # optional callback to monitor training
train!(sol, 10_000; cb=cb)
plot(cb, lw=2)

Finding Kuhn Poker Nash Equilibrium with external sampling & discounting

Kuhn Poker Implementation & Game Definition Tutorial

game = Kuhn()
sol = ESCFRSolver(game; method=Discount(α=1.0, β=1.0, γ=1.0))
cb = CFR.ExploitabilityCallback(sol)
train!(sol, 100_001; cb=cb)

hist = cb.hist
exp_idxs = 10 .^(0:5) .+ 1
plot(
    hist.x[exp_idxs], hist.y[exp_idxs];
    xscale = :log10, lw=2, label="",
    xlabel = "Training Iterations",
    ylabel = "Exploitability"
)