Adding algorithm TD Learning with N-Tuple Networks for 2048

[Jazeem]

An algorithm that can win at 2048 without using tree search or hand crafted state evaluator functions.

Currently the code is somewhat coupled with 2048. It is possible to decouple it, if that is valuable for open_spiel. Will have to find a way to move n_tuples into game.

Have modified the game 2048 to consider a move that doesn't change the board as invalid

Implementation is similar to https://github.com/moporgic/TDL2048-Demo

[lanctot]

Cool! We had quite a big refactor committed to the master branch yesterday. Could you pull changes from master and commit the merge (or rebase)?

We basically just moved all the games into subdirectories.

I cannot run the tests while there are conflicts.

[Jazeem]

Sure. Rebased now 👍🏻

[lanctot]

Can you lint your code by following Step 9 in the Adding a New Game on this page? https://github.com/deepmind/open_spiel/blob/master/docs/developer_guide.md

It greatly reduced the load on our end to have the code formatted as required by Google python standards.

E.g. indentation should be two spaces, not four. Etc.

[Jazeem]

@lanctot

• Used pylint and formatted the code
• Introduced a class NTupleNetwork and added a brief description about what N-Tuples are and a citation to a IEEE paper that this implementation is based on
• The output of the code is very straightforward similar to breakthrough_dqn.py in that for every 1000 runs; the code prints what's the average score reached, max score and largest tile that was unlocked. Running the code you can see the average scores increasing indicating that learning is happening. Once the 2048 tile is unlocked, the agent is able to win the game

[lanctot]

Hi @Jazeem,

Can you respond to the conversations above (or resolve them now if they are resolved)

[Jazeem]

@lanctot

Have resolved them