ML starter bot error when training on replays

[djbyrne]

I have just downloaded several .hlt replay files and when I run the train.py file I am receiving this error

Load Replay: training/replay-20180824-153109-0400-1535139069-32-32.hlt
Traceback (most recent call last):
File "train.py", line 6, in
m.train_on_files('training', 'TheDuck314')
File "/Users/donalbyrne/Workspace/Halite/SVM/model.py", line 43, in train_on_files
game_data = parse.parse_replay_folder(replay_folder, player_name, max_files=self.MAX_FILES)
File "/Users/donalbyrne/Workspace/Halite/SVM/parse.py", line 85, in parse_replay_folder
replay_buffer.append(parse_replay_file(os.path.join(folder_name, file_name), player_name))
File "/Users/donalbyrne/Workspace/Halite/SVM/parse.py", line 15, in parse_replay_file
data = json.loads(zstd.loads(f.read()))
File "/Users/donalbyrne/anaconda/envs/DQN_35/lib/python3.5/json/init.py", line 312, in loads
s.class.name))
TypeError: the JSON object must be str, not 'bytes'

Is this because the replay is not in JSON format?

Any help you can give would be appreciated

[snaar]

It's possibly just compressed - compression used is facebook's zstd, so you can decompress it yourself using tools they provide. Alternatively, use the tool we have to download replays in bulk - it decompresses itself - https://forums.halite.io/t/build-a-machine-learning-bot/96

[snaar]

Or alternatively, someone can add support for compressed replays in the SVM bot :P

[djbyrne]

I was able to get it working by changing the following lines in parse.py

line 15: data = json.loads(zstd.loads(f.read()).decode('utf-8'))
line 21: player = [p for p in data['players'] if p['name'] == player_name][0]

[snaar]

Ah ok, so it was already decompressing.

[Janzert]

Almost certainly the problem is because you are using python 3.5, json.loads was changed in 3.6 to accept either str, bytes, or bytearray.

From the python docs:

Changed in version 3.6: s can now be of type bytes or bytearray. The input encoding should be UTF-8, UTF-16 or UTF-32.

[djbyrne]

Almost certainly the problem is because you are using python 3.5, json.loads was changed in 3.6 to accept either str, bytes, or bytearray.

From the python docs:

Changed in version 3.6: s can now be of type bytes or bytearray. The input encoding should be UTF-8, UTF-16 or UTF-32.

Thanks Janzert, I'll try that

[djbyrne]

Changing to 3.6 worked for me, thanks for the help

[snaar]

Opened #163 to add version check.