The model is not improving.

[Cranial-XIX]

Hi, after I cloned your repo and ran the dagger.py, the model doesn't seem to improve. In fact, it stops at 28 iterations every time. I didn't change any parameter in your code. So could you please take a look at it? Also, how did you get those .gif on the README? Are those just from the TORCS visualization?

[james-mchugh]

Did you change the settings for Torcs to make the resolution 64x64 and set the default view mode to driving mode? The settings to be used for the experiment can be found in the gym-torcs repository.

Also, try changing the activation function to linear and reducing the learning rate. I believe this this is a regression problem, but I think tanh is better suited for classification problems. See this issue.

[james-mchugh]

Hello. I have some new updates. I finally got the simulation to run after tweaking it a bit.

Along with the previous changes that I mentioned, I set the kernel initialization to uniform. After messing with the simulation a bit, I realized that once in a while, the simulation would run properly and converge. I attributed this to the way the parameters were being initialized, which I believe is a random normal initialization by default. I changed this to a random uniform instead and it seems to be working a lot better.

I also added a function to save the images from each run as a gif. I forked @avisingh599's repo, and my version can be found here.