🟢: thoroughly-tested. In many cases, we verified against known values and/or reproduced results from papers.
🔶: implemented but lightly tested.
❌: known issues (see notes below and code for details).
- A single player game where player aims to create a 2048 tile by merging other tiles.
- Numbers on a grid.
- Modern game.
- Non-deterministic.
- Perfect information.
- 1 player.
- Github
- Move pieces on a board trying to block opponents from moving.
- Pieces on a grid.
- Modern game.
- Deterministic.
- Perfect information.
- 2 players.
- Wikipedia
- Agent plays classic games from Gym's Atari Environments, such as Breakout.
- Single player.
- Most games are non-deterministic.
- Perfect information.
- Players move their pieces through the board based on the rolls of dice.
- Idiosyncratic format.
- Traditional game.
- Non-deterministic.
- Perfect information.
- 2 players.
- Wikipedia
- Agents negotiate for items in a pool with different (hidden) valuations.
- Research game.
- Non-deterministic (randomized pool and valuations).
- Imperfect information.
- 2 players.
- Lewis et al. '17, DeVault et al. '15
- Players place ships and shoot at each other in turns.
- Pieces on a board.
- Traditional game.
- Deterministic.
- Imperfect information.
- 2 players.
- Good for correlated equilibria.
- Farina et al. '19, Correlation in Extensive-Form Games: Saddle-Point Formulation and Benchmarks. Based on the original game (wikipedia)
- Simplified version of blackjack, with only HIT/STAND moves.
- Traditional game.
- Non-deterministic.
- Imperfect information.
- 1 player.
- Wikipedia
- Most simple version of dominoes.
- Consists of 28 tiles, featuring all combinations of spot counts (also called pips or dots) between zero and six.
- Traditional game.
- Non-deterministic.
- Imperfect information.
- 2 players.
- Wikipedia
- Simplified chess using only pawns.
- Pieces on a grid.
- Modern game.
- Deterministic.
- Perfect information.
- 2 players.
- Wikipedia
- A card game where players compete in pairs.
- Card game.
- Traditional game.
- Non-deterministic.
- Imperfect information.
- 4 players.
- Wikipedia
- Players score points by forming specific sets with the cards in their hands.
- Card game.
- Research game.
- Non-deterministic.
- Imperfect information.
- 2 players.
- Wikipedia
- Agent must move horizontally to 'catch' a descending ball. Designed to test basic learning.
- Agent on a grid.
- Research game.
- Non-deterministic.
- Perfect information.
- 1 players.
- Mnih et al. 2014, Recurrent Models of Visual Attention,
Osband et al '19, Behaviour Suite for Reinforcement Learning, Appendix A
- Players move pieces around the board with the goal of eliminating the opposing pieces.
- Pieces on a grid.
- Traditional game.
- Deterministic.
- Perfect information.
- 2 players.
- Wikipedia
- Agent must find goal without falling off a cliff. Designed to demonstrate exploration-with-danger.
- Agent on a grid.
- Research game.
- Deterministic.
- Perfect information.
- 1 players.
- Sutton et al. '18, page 132
- Simplified checkers, where tokens can capture neighbouring tokens. Designed to be amenable to combinatorial analysis.
- Pieces on a grid.
- Research game.
- Deterministic.
- Perfect information.
- 2 players.
- Wikipedia
- Agents must collect their and their collaborator's tokens while avoiding a third kind of token. Designed to test divining of collaborator's intentions
- Agents on a grid.
- Research game.
- Non-deterministic.
- Imperfect information (all players see the grid and their own preferences, but not the preferences of other players).
- 2 players.
- Raileanu et al. '18, Modeling Others using Oneself in Multi-Agent Reinforcement Learning
- Agents negotiations for chips that they they play on a colored grid to move closer to the goal.
- Agents on a grid.
- Research game.
- Non-deterministic (randomized board & chip configuration).
- Imperfect information.
- 3 players.
- Ya'akov et al. '10, Fecici & Pfeffer '08, de Jong et al. '11
- Players drop tokens into columns to try and form a pattern.
- Tokens on a grid.
- Traditional game.
- Deterministic.
- Perfect information.
- 2 players.
- Wikipedia
- Agents must collaborate to push a box into the goal. Designed to test collaboration.
- Agents on a grid.
- Research game.
- Deterministic.
- Perfect information.
- 2 players.
- Seuken & Zilberstein '12, Improved Memory-Bounded Dynamic Programming for Decentralized POMDPs
- Players move pieces around the board with the goal of eliminating the opposing pieces.
- Pieces on a grid.
- Traditional game.
- Deterministic.
- Perfect information.
- 2 players.
- Wikipedia
- Players put lines between dots to form boxes to get points.
- Traditional game.
- Deterministic.
- Perfect information.
- 2 players.
- Wikipedia
- A precursor of UNO (see here).
- Players try to match the rank or suit of the previous played card.
- Eights are viewed as wild cards.
- In an alternative version, special cards such as skip, reverse, draw-two are permitted.
- Nondeterministic.
- Imperfect information.
-
=2 players.
- Wikipedia
- Hex, except the opponent's tokens are hidden. (Imperfect-information version)
- Uses tokens on a hex grid.
- Research game.
- Deterministic.
- Imperfect information.
- 2 players.
- Agent must explore to find reward (first version) or penalty (second version). Designed to test exploration.
- Agent on a grid.
- Research game.
- Deterministic.
- Perfect information.
- 1 players.
- Osband et al. '17, Deep Exploration via Randomized Value Functions
- A three-player games where one player (dizhu) plays against a team of two (peasants).
- Uses a 54-card deck.
- Non-deterministic.
- Imperfect information.
- Three players.
- Wikipedia
- Trick-taking card game where players compete in pairs.
- Card game.
- Traditional game.
- Non-deterministic.
- Imperfect information.
- 4 players.
- Wikipedia
- Agents submit bids simultaneously; highest bid wins, and that's the price paid.
- Idiosyncratic format.
- Research game.
- Non-deterministic.
- Imperfect, incomplete information.
- 2-10 players.
- Wikipedia
- Players score points by forming specific sets with the cards in their hands.
- Card game.
- Traditional game.
- Non-deterministic.
- Imperfect information.
- 2 players.
- Wikipedia
- Players place tokens on the board with the goal of encircling territory.
- Tokens on a grid.
- Traditional game.
- Deterministic.
- Perfect information.
- 2 players.
- Wikipedia
- Players bid with their cards to win other cards.
- Card game.
- Traditional game.
- Non-deterministic.
- Imperfect information.
- 2-10 players.
- Wikipedia
- Players can see only other player's pieces, and everyone must cooperate to win.
- Idiosyncratic format.
- Modern game.
- Non-deterministic.
- Imperfect information.
- 2-5 players.
- Wikipedia and Bard et al. '19, The Hanabi Challenge: A New Frontier for AI Research
- Implemented via Hanabi Learning Environment
- Players add tokens to a hex grid to try and form a winning structure.
- Tokens on a hex grid.
- Modern game.
- Deterministic.
- Perfect information.
- 2 players.
- Wikipedia
- A card game where players try to avoid playing the highest card in each round.
- Card game.
- Traditional game.
- Non-deterministic.
- Imperfect information.
- 3-6 players.
- Wikipedia
- Players add tokens to a hex grid to try and link opposite sides of the board.
- Uses tokens on a hex grid.
- Modern game.
- Deterministic.
- Perfect information.
- 2 players.
- Wikipedia
- Hex, the full story by Ryan Hayward and Bjarne Toft
- Chess with opponent's pieces unknown. Illegal moves have no effect - it remains the same player's turn until they make a legal move.
- Traditional chess variant, invented by Henry Michael Temple in 1899.
- Deterministic.
- Imperfect information.
- 2 players.
- Wikipedia
- Monte Carlo tree search in Kriegspiel
- Game-Tree Search with Combinatorially Large Belief States, Parker 2005
- Simplified poker amenable to game-theoretic analysis.
- Cards with bidding.
- Research game.
- Non-deterministic.
- Imperfect information.
- 2 players.
- Wikipedia
- Agents see a local part of the grid, and attempt to tag each other with beams.
- Agents on a grid.
- Research game.
- Non-deterministic.
- Imperfect information.
- 2 players.
- Leibo et al. '17, Lanctot et al. '17
- Simplified poker amenable to game-theoretic analysis.
- Cards with bidding.
- Research game.
- Non-deterministic.
- Imperfect information.
- 2 players.
- Southey et al. '05, Bayes’ bluff: Opponent modelling in poker
- Receiver must choose an action dependent on the sender's hidden state. Designed to demonstrate the use of conventions.
- Idiosyncratic format.
- Research game.
- Non-deterministic.
- Imperfect information.
- 2 players.
- Wikipedia
- Players bid and bluff on the state of all the dice together, given only the state of their dice.
- Dice with bidding.
- Traditional game.
- Non-deterministic.
- Imperfect information.
- 2 players.
- Wikipedia
- Players bid and bluff on the state of all hands, given only the state of their hand.
- Cards with bidding.
- Traditional game.
- Non-deterministic.
- Imperfect information
- 2 or more players.
- Wikipedia
- Players roll dice to move their pegs toward their home row while throwing other players' pegs to the out area.
- Traditional game.
- Non-deterministic.
- Perfect information.
- 2-4 players.
- Wikipedia
- Players take turns sowing beans on the board and try to capture more beans than the opponent.
- Idiosyncratic format.
- Traditional game.
- Deterministic.
- Perfect information.
- 2 players.
- Wikipedia
- Agents must take the ball to their goal, and can 'tackle' the opponent by predicting their next move.
- Agents on a grid.
- Research game.
- Non-deterministic.
- Imperfect information.
- 2 players.
- Littman '94, Markov games as a framework for multi-agent reinforcement learning,
He et al. '16, Opponent Modeling in Deep Reinforcement Learning
- Players must predict and match/oppose another player. Designed to have an unstable Nash equilibrium.
- Idiosyncratic format.
- Research game.
- Deterministic.
- Imperfect information.
- 3 players.
- "Three problems in learning mixed-strategy Nash equilibria"
- Representative player chooses at each node where they go. They has an origin, a destination and a departure time and chooses their route to minimize their travel time. Time spent on each link is a function of the distribution of players on the link when the player reaches the link.
- Network with choice of route.
- Research game.
- Mean-field (with a unique player).
- Explicit stochastic game (only for initial node).
- Perfect information.
- Cabannes et. al. '21, Solving N-player dynamic routing games with congestion: a mean field approach.
- Players are uniformly distributed and are then incentivized to gather at the same point (The lower the distanbce wrt. the distribution mean position, the higher the reward). A mean-reverting term pushes the players towards the distribution, a gaussian noise term perturbs them. The players' actions alter their states linearly (alpha * a * dt) and the cost thereof is quadratic (K * a^2 * dt), hence the name. There exists an exact, closed form solution for the fully continuous version of this game.
- Research game.
- Mean-field (with a unique player).
- Explicit stochastic game (only for initial node).
- Perfect information.
- [Perrin & al. 2019 (https://arxiv.org/abs/2007.03458)]
- A single player game where player aims to maximize lines drawn on a grid, under certain limitations.
- Uses tokens on a grid.
- Traditional game.
- Deterministic
- Perfect information.
- 1 player.
- Wikipedia
- Agents with different utilities must negotiate an allocation of resources.
- Idiosyncratic format.
- Research game.
- Non-deterministic.
- Imperfect information.
- 2 players.
- Lewis et al. '17, Cao et al. '18
- Two agents take objects from distinct piles trying to either avoid taking the last one or take it. Any positive number of objects can be taken on each turn given they all come from the same pile.
- Traditional mathematical game.
- Deterministic.
- Perfect information.
- 2 players.
- Wikipedia
- Two players put and move stones on the board to try to form mills (three adjacent stones in a line) to capture the other player's stones.
- Traditional game.
- Deterministic.
- Perfect information.
- 2 players.
- Wikipedia
- A card game where players try to win exactly a declared number of tricks.
- Card game.
- Traditional game.
- Non-deterministic.
- Imperfect information.
- 3-7 players.
- Wikipedia
- Players must repeatedly bid to push a token off the other side of the board.
- Idiosyncratic format.
- Traditional game.
- Deterministic.
- Imperfect information.
- 2 players.
- Buro, 2004. Solving the oshi-zumo game
Bosansky et al. '16, Algorithms for Computing Strategies in Two-Player Simultaneous Move Games
- Players redistribute tokens from their half of the board to capture tokens in the opponent's part of the board.
- Idiosyncratic format.
- Traditional game.
- Deterministic.
- Perfect information.
- 2 players.
- Wikipedia
- Agents must move to their desitnation.
- Agents on a grid. Single-agent game is the classic examples from Sutton & Barto.
- Research game.
- Non-deterministic (in multiagent, collisions resolved by chance nodes).
- Perfect information.
- 1-10 players.
- Similar games appeared in Austerweil et al. '15, Greenwald & Hall '03, and Littman '01.
- Players place tokens on the board, then rotate part of the board to a new orientation.
- Uses tokens on a grid.
- Modern game.
- Deterministic.
- Perfect information.
- 2 players.
- Wikipedia
- Go, except the opponent's stones are hidden. The analogue of Kriegspiel for Go.
- Research game.
- Deterministic.
- Imperfect information.
- 2 players.
- Cazenave '05, A Phantom Go Program
- Tic-tac-toe, except the opponent's tokens are hidden. Designed as a simple, imperfect-information game.
- Uses tokens on a grid.
- Research game.
- Deterministic.
- Imperfect information.
- 2 players.
- Auger '11, Multiple Tree for Partially Observable Monte-Carlo Tree Search,
Lisy '14, Alternative Selection Functions for Information Set Monte Carlo Tree Search,
Lanctot '13
- Each player rolls a dice until they get a 1 or they 'hold'; the rolled total is added to their score.
- Dice game.
- Traditional game.
- Non-deterministic.
- Perfect information.
- 2-10 players.
- Wikipedia
- Players decide on whether to cooperate or defect given a situation with different payoffs.
- Simultaneous.
- Traditional game.
- Deterministic.
- Perfect Information.
- 2 players.
- Wikipedia
- Players bet on whether their hand of cards plus some communal cards will form a special set.
- Cards with bidding.
- Traditional game.
- Non-deterministic.
- Imperfect information.
- 2-10 players.
- Wikipedia
- Implemented via ACPC.
- ❌ Known issues: see issues #778, #1033, and #1042.
- Each turn, players can either move their agent or add a small wall to the board.
- Idiosyncratic format.
- Modern game.
- Deterministic.
- Perfect information.
- 2-4 players. (Note, from Wikipedia: "Though it can be played with 3 players, it's advised against. Since the 3rd player doesn't have player on the opposite side, they have an advantage.")
- Wikipedia
- ❌ Known issues: see #1158.
- Chess with opponent's pieces unknown, with sensing moves.
- Chess variant, invented by John Hopkins University Applied Physics Lab. Used in NeurIPS competition and Hidden Information Game Competition.
- Deterministic.
- Imperfect information.
- 2 players.
- JHU APL Main site
- Markowitz et al. '18, On the Complexity of Reconnaissance Blind Chess
- Newman et al. '16, Reconnaissance blind multi-chess: an experimentation platform for ISR sensor fusion and resource management
- ❌ Known issues: see #811.
- Players choose at each node where they go. They have an origin, a destination and a departure time and choose their route to minimize their travel time. Time spent on each link is a function of the number of players on the link when the player reaches the link.
- Network with choice of route.
- Research game.
- Simultaneous.
- Deterministic.
- Perfect information.
- Any number of players.
- Cabannes et. al. '21, Solving N-player dynamic routing games with congestion: a mean field approach.
- Bargaining game.
- Deterministic.
- Imperfect information.
- 2 players.
- Good for correlated equilibria.
- Farina et al. '19, Correlation in Extensive-Form Games: Saddle-Point Formulation and Benchmarks.
- Based on the board game "Sheriff of Nottingham" (bbg)
- Trick-based card game with bidding.
- Traditional game.
- Non-deterministic.
- Imperfect information.
- 3-4 players.
- Wikipedia
- Luštrek et al. 2003, A program for playing Tarok
- Each turn, players bid to compete against the other two players.
- Cards with bidding.
- Traditional game.
- Non-deterministic.
- Imperfect information.
- 3 players.
- Wikipedia
- A single-player card game.
- Card game.
- Traditional game.
- Non-deterministic.
- Imperfect information.
- 1 players.
- Wikipedia and Bjarnason et al. '07, Searching solitaire in real time
- Players place tokens to try and form a pattern.
- Uses tokens on a grid.
- Traditional game.
- Deterministic.
- Perfect information.
- 2 players.
- Wikipedia
- Simplified Bridge with fewer cards and tricks.
- Cards with bidding.
- Research game.
- Non-deterministic.
- Imperfect information.
- 2, 4 players.
- See implementation for details.
- Simplified Hanabi with just two turns.
- Idiosyncratic format.
- Research game.
- Non-deterministic.
- Imperfect information.
- 2-10 players.
- Foerster et al 2018, Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learning
- Players with different utilities and items communicate and then trade.
- Idiosyncratic format.
- Research game.
- Non-deterministic.
- Imperfect information.
- 2 players.
- A simple emergent communication game based on trading.
- Players try and form a pattern in local boards and a meta-board.
- Uses tokens on a grid.
- Deterministic.
- Perfect information.
- 2 players.
- Wikipedia
- Classic coalitional game.
- Players each have a weight w_i, and there is a quota q.
- Denote p the binary vector representing a coalition over n players. The utility is 1 is p dot w >= q, 0 otherwise.
- n players.
- Chalkiadakis, Elkind, & Wooldridge '12
- Players place tokens to try and connect sides of a triangular board.
- Tokens on hex grid.
- Modern game.
- Deterministic.
- Perfect information.
- 2 players.
- Wikipedia