[Bug Report] CartPoleVectorEnv resets one step to early, not following the new VectorEnv API #914
Closed
1 task done
Labels
bug
Something isn't working
Describe the bug
Hi,
according to #785, vector environments are supposed to return the final observation when the current episode is done, instead of immediately returning the first observation of the new episode.
CartPoleVectorEnv
currently does not do this and instead resets its done sub-environments immediately. To fix this issue, the reset of the sub-environments must be delayed, as inSyncVectorEnv
.Best,
Tim
Code example
System info
No response
Additional context
No response
Checklist
The text was updated successfully, but these errors were encountered: