train mineral shard example is not compatible with baselines library

[snurkabill]

I can't get started train_minteral_shards.py example. Getting this error:

$ python train_mineral_shards.py
Traceback (most recent call last):
File "train_mineral_shards.py", line 14, in
from common.vec_env.subproc_vec_env import SubprocVecEnv
File "D:\devel\repositories\pysc2-examples\common_init_.py", line 1, in
from baselines.common.console_util import *
ImportError: No module named 'baselines.common.console_util'

btw, I am using windows, can't it be problem? I've tried build baselines from repository, but I've failed on atari.py dependency (not very compatible with windows). baselines0.1.4 or lower libraries can be installed using pip, but seem to be not sufficient for example. Please help how to proceed.

[chris-chris]

You need to install baselines library from the baselines Github repo.

pip install git+https://github.com/openai/baselines

Try this!

[snurkabill]

ok, I've switched to another machine with linux and I was able to install baselines from repository. (baselines with version 0.1.4 does not work). Now I am getting error while running this script:

python3 train_mineral_shards.py --algorithm=deepq --prioritized=True --dueling=True --timesteps=2000000 --exploration_fraction=0.2 --num_cpu=4 --lr=0.0005
algorithm : deepq
timesteps : 2000000
exploration_fraction : 0.2
prioritized : True
dueling : True
num_cpu : 4
lr : 0.0005
Traceback (most recent call last):
  File "train_mineral_shards.py", line 289, in <module>
    main()
  File "train_mineral_shards.py", line 97, in main
    visualize=True) as env:
  File "/home/snurkabill/.local/lib/python3.5/site-packages/pysc2/env/sc2_env.py", line 116, in __init__
    raise ValueError("All arguments must be passed as keyword arguments.")
ValueError: All arguments must be passed as keyword arguments.

also for example:

algorithm : acktr
timesteps : 2000000
exploration_fraction : 0.5
prioritized : True
dueling : True
num_cpu : 4
lr : 0.0005
Process Process-1:
Traceback (most recent call last):
  File "/usr/lib/python3.5/multiprocessing/process.py", line 249, in _bootstrap
    self.run()
  File "/usr/lib/python3.5/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/home/snurkabill/dev/repositories/pysc2-examples/common/vec_env/subproc_vec_env.py", line 15, in worker
    step_mul=1) as env:
  File "/home/snurkabill/.local/lib/python3.5/site-packages/pysc2/env/sc2_env.py", line 116, in __init__
    raise ValueError("All arguments must be passed as keyword arguments.")
ValueError: All arguments must be passed as keyword arguments.
Process Process-2:
Traceback (most recent call last):
  File "/usr/lib/python3.5/multiprocessing/process.py", line 249, in _bootstrap
    self.run()
  File "/usr/lib/python3.5/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/home/snurkabill/dev/repositories/pysc2-examples/common/vec_env/subproc_vec_env.py", line 15, in worker
    step_mul=1) as env:
  File "/home/snurkabill/.local/lib/python3.5/site-packages/pysc2/env/sc2_env.py", line 116, in __init__
    raise ValueError("All arguments must be passed as keyword arguments.")
ValueError: All arguments must be passed as keyword arguments.
Process Process-3:
Traceback (most recent call last):
  File "/usr/lib/python3.5/multiprocessing/process.py", line 249, in _bootstrap
    self.run()
  File "/usr/lib/python3.5/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/home/snurkabill/dev/repositories/pysc2-examples/common/vec_env/subproc_vec_env.py", line 15, in worker
    step_mul=1) as env:
  File "/home/snurkabill/.local/lib/python3.5/site-packages/pysc2/env/sc2_env.py", line 116, in __init__
    raise ValueError("All arguments must be passed as keyword arguments.")
ValueError: All arguments must be passed as keyword arguments.
Process Process-4:
Traceback (most recent call last):
  File "/usr/lib/python3.5/multiprocessing/process.py", line 249, in _bootstrap
    self.run()
  File "/usr/lib/python3.5/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/home/snurkabill/dev/repositories/pysc2-examples/common/vec_env/subproc_vec_env.py", line 15, in worker
    step_mul=1) as env:
  File "/home/snurkabill/.local/lib/python3.5/site-packages/pysc2/env/sc2_env.py", line 116, in __init__
    raise ValueError("All arguments must be passed as keyword arguments.")
ValueError: All arguments must be passed as keyword arguments.

Can you point me forward one more time, please?

[snurkabill]

Hi, @chris-chris . I've tried modify your examples but I somehow failed. Could you please provide solution to my problem? I would be very grateful :)

[ShadowDancer]

@snurkabill
Try adding
_only_use_kwargs=False
to constructor "with sc2_env.SC2Env("
Idk what is purpose of this code >.>

[chris-chris]

@snurkabill

Can you pull the latest code and try it again?
I fixed this issue recently :)

[mcdavid109]

I still got that errors:
Traceback (most recent call last):
File "train_mineral_shards.py", line 13, in
from common.vec_env.subproc_vec_env import SubprocVecEnv
File "/home/howard/pysc2-examples/common/init.py", line 1, in
from baselines.common.console_util import *
ImportError: No module named 'baselines.common.console_util'

[ghost]

@mcdavid109

pip install git+https://github.com/openai/baselines

same thing "ImportError: No module named 'baselines.common.console_util'"

so, I tried

1. Step : git clone https://github.com/openai/baselines.git
2. Step : pip install -e . (in baselines directory)

and found 'baselines.common.console_util'

Environment : win10 + bash.. // bash app is very honey!

[Faur]

I (Ubuntu 16.04) get this error

Failed building wheel for mpi4py

Both when I do

pip install git+https://github.com/openai/baselines

and as @d4r6

[Faur]

Solved the mpi4py
issue with this
https://geeksww.com/tutorials/operating_systems/linux/installation/downloading_compiling_and_installing_cmake_on_linux.php

Then I had a problem with installing atari-py, which i solved using this
openai/gym#218

[mushroom1116]

@Faur
I had this "mpi4py" problem too and I solved it this way:
1 sudo apt-get install libopenmpi-dev openmpi-bin openmpi-doc
2 pip install mpi4py

[gokimsky]

I have the same exact issue with snurkabill.
ImportError: No module named 'baselines.common.console_util'
I am using win10. Do I have to change to Linux?

[brean]

@gokimsky no. I have the example running. I am using anaconda with a tensorflow-gpu installation on windows 10.
If you install the current atari-py first from their git repo and then baselines you do not need mpi:

git clone [email protected]:openai/atari-py.git
cd atari-py
pip install -e .
cd ..
git clone [email protected]:openai/baselines.git
cd baselines
pip install -e .
cd ..

The second problem is that the baselines API has changed, so the code here needs some changes.
I got the defeat_zerglings and enjoy_mineral_shads files running in my fork for now, take a look at my changes here:
master...brean:baseline_changes
(I am not that experienced with deep learning yet so I will not directly provide a pull request and let you guys take a look at my code first, so feel free to copy stuff from it and merge it with this repo).

[lml519]

when i run the

python3 train_mineral_shards.py --algorithm=a2c
I got a similar problem,but this is the problem of my tensorflow.
The output is that:

Traceback (most recent call last):
File "train_mineral_shards.py", line 5, in
from baselines import deepq
File "/home/limenglin519/pysc2-examples/baselines/baselines/deepq/init.py", line 1, in
from baselines.deepq import models # noqa
File "/home/limenglin519/pysc2-examples/baselines/baselines/deepq/models.py", line 2, in
import tensorflow.contrib.layers as layers
ImportError: No module named 'tensorflow.contrib'
I wuold be grateful ,if u can help me to solve this.