Change end-of-episode in CarRacing to termination as opposed to truncation #813
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There's been some heated debate about this, I'm still open to have my mind changed, but imo with the new termination-truncation split, this is a clear example of a termination as opposed to truncation.
Truncation is meant to represent hitting the "externally imposed" time limit as opposed to actually reaching a terminal state. When the car does the laps that it needs, the episode should be terminated.
To give some more information, I also added an info entry describing the reason for termination (whether or not the lap was finished)
This also requires a version bump, since implementations that correctly handle termination/truncation may converge to different policies