experiment.ipynb

Kernel: Python 3

Counterfactual Regret Minimization (CFR) on Kuhn Poker

This is an experiment learning to play Kuhn Poker with Counterfactual Regret Minimization CFR algorithm.

Install the labml-nn package

%%capture
!pip install labml-nn

Imports

from labml import experiment, analytics
from labml_nn.cfr.analytics import plot_infosets
from labml_nn.cfr.kuhn import Configs
from labml_nn.cfr.infoset_saver import InfoSetSaver

Create an experiment, we only write tracking information to sqlite to speed things up. Since the algorithm iterates fast and we track data on each iteration, writing to other destinations such as Tensorboard can be relatively time consuming. SQLite is enough for our analytics.

experiment.create(name='kuhn_poker', writers={'sqlite'})

Initialize configurations

conf = Configs()

Set experiment configurations and assign a configurations dictionary to override configurations

experiment.configs(conf, {'epochs': 1_000_000})

Start the experiment and run the training loop.

# Start the experiment
with experiment.start():
    conf.cfr.iterate()

inds = analytics.runs(experiment.get_uuid())

# dir(inds)

plot_infosets(inds['average_strategy.*'], width=600, height=500).display()

analytics.scatter(inds.average_strategy_Q_b, inds.average_strategy_Kb_b,
                  width=400, height=400)