Minimizing Weighted Counterfactual Regret with Optimistic Online Mirror Descent

Minimizing Weighted Counterfactual Regret with Optimistic Online Mirror Descent
Hang Xu, Kai Li^#, Bingyun Liu, Haobo Fu, Qiang Fu, Junliang Xing^#, Jian Cheng
IJCAI 2024 (Oral)

Install PDCFRPlus

Install miniconda3 from the official website and run the following script:

bash scripts/install.sh

Test PDCFRPlus

We use games implemented by OpenSpiel [1] and PokerRL [2]. Run the following script to assess the performance of CFR variants on testing games. The results are saved in the folder results.

conda activate PDCFRPlus
python scripts/parallel_run.py --algo CFRPlus
python scripts/parallel_run.py --algo LinearCFR
python scripts/parallel_run.py --algo DCFR
python scripts/parallel_run.py --algo PCFRPlus --gamma=2
python scripts/parallel_run.py --algo PCFRPlus --gamma=5
python scripts/parallel_run.py --algo DCFRPlus --gamma=4 --alpha=1.5
python scripts/parallel_run.py --algo PDCFRPlus --gamma=5 --alpha=2.3

python scripts/parallel_run.py --algo PIDCFR --average=False
find results -type d -regex ".*/*/PIDCFR"  -exec rm -rf {} +
find results -type d -regex ".*/*/PIDCFR/38-3"  -exec rm -rf {} +
python scripts/run.py with game_name="KuhnPoker" algo_name="PIDCFR" iterations=5000 save_log=True
export PYTHONPATH=/home/caihuaiguang/DSG/pytorch-shapley-cam:$PYTHONPATH
python scripts/draw.py 

python scripts/run.py with game_name="SmallMatrix" algo_name="CFR" iterations=5000 save_log=True average=False

python scripts/run.py with game_name="GoofSpielImp4" algo_name="PIDCFR" iterations=5000 save_log=True average=False

python scripts/run.py with game_name="Subgame3" algo_name="PDCFRPlus" iterations=5000 save_log=True

References

[1] Lanctot, M.; Lockhart, E.; Lespiau, J.-B.; Zambaldi, V.; Upadhyay, S.; P´erolat, J.; Srinivasan, S.; Timbers, F.; Tuyls, K.; Omidshafiei, S.; Hennes, D.; Morrill, D.; Muller, P.; Ewalds, T.; Faulkner, R.; Kram´ar, J.; Vylder, B. D.; Saeta, B.; Bradbury, J.; Ding, D.; Borgeaud, S.; Lai, M.; Schrittwieser, J.; Anthony, T.; Hughes, E.; Danihelka, I.; and Ryan-Davis, J. 2019. OpenSpiel: A Framework for Reinforcement Learning in Games. CoRR, abs/1908.09453.

[2] Steinberger, E. 2019. PokerRL. https://github.com/TinkeringCode/PokerRL.