Check the determinism of env.reset(seed=42); env.reset()

[qgallouedec]

Description

Fixes #1084 (see the issue for the description)

Type of change

Please delete options that are not relevant.

• Documentation only change (no code changed)
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Checklist:

• I have run the pre-commit checks with pre-commit run --all-files (see CONTRIBUTING.md instructions to set it up)
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation

-> No need, already documented

• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works ->

-> No, but catching the non-determinism of env.reset was not part of the test, so I let this for further improvement coverage.
You can still check that the new feature works with the code provided in #1084 (the error is now catched by the env checker):

Traceback (most recent call last):
  File "/Users/quentingallouedec/Gymnasium/t.py", line 32, in <module>
    check_env(gym.make("MyEnv-v0"))  # Passes but:
  File "/Users/quentingallouedec/Gymnasium/gymnasium/utils/env_checker.py", line 412, in check_env
    check_reset_seed_determinism(env)
  File "/Users/quentingallouedec/Gymnasium/gymnasium/utils/env_checker.py", line 116, in check_reset_seed_determinism
    assert data_equivalence(
AssertionError: Using `env.reset(seed=123)` then `env.reset()` is non-deterministic as the observations are not equivalent.

• New and existing unit tests pass locally with my changes

[qgallouedec]

Tests pass locally, not sure why they fail here. If you can take a look @pseudo-rnd-thoughts

[pseudo-rnd-thoughts]

Strange this fails for me locally

[qgallouedec]

It must be good now.

[pseudo-rnd-thoughts]

Could you provide an example environment that fails the previous version that this PR fixes.
I was imagining, we loop over several seed values rather than just copy and paste the code which is what seems to happen

[qgallouedec]

See the example environment from #1084

import gymnasium as gym
from gymnasium.utils.env_checker import check_env
import random


class MyEnv(gym.Env):
    def __init__(self):
        super().__init__()
        self.observation_space = gym.spaces.Discrete(5)
        self.action_space = gym.spaces.Discrete(5)

    def reset(self, seed=None, options=None):
        super().reset(seed=seed)
        if seed is not None:
            obs = self.np_random.integers(5)
        else:
            # generate a random observations, but not based on the inner rng -> bad => non deterministic
            obs = int(random.random() * 5)
        return obs, {}

    def step(self, action):
        obs = self.np_random.integers(5)
        return obs, 0, False, False, {}


register = gym.register(
    id="MyEnv-v0",
    entry_point=MyEnv,
    max_episode_steps=1000,
)

check_env(gym.make("MyEnv-v0"))

Before: no error, now:

After:

Traceback (most recent call last):
  File "/Users/quentingallouedec/Gymnasium/t.py", line 32, in <module>
    check_env(gym.make("MyEnv-v0"))
  File "/Users/quentingallouedec/Gymnasium/gymnasium/utils/env_checker.py", line 412, in check_env
    check_reset_seed_determinism(env)
  File "/Users/quentingallouedec/Gymnasium/gymnasium/utils/env_checker.py", line 116, in check_reset_seed_determinism
    assert data_equivalence(
AssertionError: Using `env.reset(seed=123)` then `env.reset()` is non-deterministic as the observations are not equivalent.

[qgallouedec]

What do you mean by "we loop over several seed values"? Isn't one enough?

[pseudo-rnd-thoughts]

Thanks for the example, using this I better understand the PR and why the looping idea that I made doesn't make sense.
LGTM