CounterfactualRegret.jl

Installation

using Pkg; Pkg.add("CounterfactualRegret")

Implemented Solvers

Solver Name	Description
CFRSolver	Vanilla CFR solver
CSCFRSolver	Chance Sampling
ESCFRSolver	External Sampling
OSCFRSolver	Outcome Sampling

Each solver takes optional kwarg method, which can be an instantiation of either Vanilla, Plus, or Discount types, which correspond to Vanilla CFR, CFR+, and discounted CFR respectively.

External Solver Packages

Name	Status
DeepCFR
ESCHER
POMDPBestResponse

Solving a Matrix Game

using CounterfactualRegret
const CFR = CounterfactualRegret
using CounterfactualRegret.Games
using Plots

game = MatrixGame([
    (1,1) (0,0) (0,0);
    (0,0) (0,2) (3,0);
    (0,0) (2,0) (0,3);
])
sol = CFRSolver(game; debug=true)
cb = CFR.ExploitabilityCallback(sol, 10) # optional callback to monitor training
train!(sol, 10_000; cb=cb)
plot(cb, lw=2)

Finding Kuhn Poker Nash Equilibrium with external sampling & discounting

Kuhn Poker Implementation & Game Definition Tutorial

game = Kuhn()
sol = ESCFRSolver(game; method=Discount(α=1.0, β=1.0, γ=1.0))
cb = CFR.ExploitabilityCallback(sol)
train!(sol, 100_001; cb=cb)

hist = cb.hist
exp_idxs = 10 .^(0:5) .+ 1
plot(
    hist.x[exp_idxs], hist.y[exp_idxs];
    xscale = :log10, lw=2, label="",
    xlabel = "Training Iterations",
    ylabel = "Exploitability"
)