Can not copy a GymEnv environment when using a LightSim2Grid backend

[EBoguslawski]

Environment

• Python version: 3.12
• Grid2op version: 1.10.5
• LightSim2Grid version: 0.9.2.post2
• System: ubuntu

Bug description

I create a GymEnv environment from a grid2op environment. If I use a LightSimBackend backend when creating the grid2op environment, then I obtain an error if I try to copy the GymEnv environment.
It works on an older environment with older versions (python 3.8, Grid2op 1.9.8, LightSim2Grid 0.7.0.post1)

How to reproduce

Code snippet

import grid2op
from grid2op.gym_compat import GymEnv
from lightsim2grid import LightSimBackend
import copy

env = grid2op.make("l2rpn_case14_sandbox", backend=LightSimBackend())
env_gym = GymEnv(env)

# This line raises an error
copy.deepcopy(env_gym)

Current output

/data/boguslawskieva/virtual_environments/venv_multi_agent/lib/python3.12/site-packages/lightsim2grid/gridmodel/from_pandapower/_aux_add_trafo.py:38: UserWarning: There were some Nan in the pp_net.trafo["tap_neutral"], they have been replaced by 0
  warnings.warn("There were some Nan in the pp_net.trafo[\"tap_neutral\"], they have been replaced by 0")
/data/boguslawskieva/virtual_environments/venv_multi_agent/lib/python3.12/site-packages/lightsim2grid/gridmodel/from_pandapower/_aux_add_trafo.py:46: UserWarning: There were some Nan in the pp_net.trafo["tap_step_percent"], they have been replaced by 0
  warnings.warn("There were some Nan in the pp_net.trafo[\"tap_step_percent\"], they have been replaced by 0")
/data/boguslawskieva/virtual_environments/venv_multi_agent/lib/python3.12/site-packages/lightsim2grid/gridmodel/from_pandapower/_aux_add_trafo.py:51: UserWarning: There were some Nan in the pp_net.trafo["tap_pos"], they have been replaced by 0
  warnings.warn("There were some Nan in the pp_net.trafo[\"tap_pos\"], they have been replaced by 0")
/data/boguslawskieva/virtual_environments/venv_multi_agent/lib/python3.12/site-packages/lightsim2grid/gridmodel/from_pandapower/_aux_add_trafo.py:70: UserWarning: There were some Nan in the pp_net.trafo["tap_step_degree"], they have been replaced by 0
  warnings.warn("There were some Nan in the pp_net.trafo[\"tap_step_degree\"], they have been replaced by 0")
/data/boguslawskieva/virtual_environments/venv_multi_agent/lib/python3.12/site-packages/lightsim2grid/gridmodel/from_pandapower/_aux_add_slack.py:56: UserWarning: LightSim will not consider the pandapower "ext_grid" as there are already generators tagged as slack bus
  warnings.warn("LightSim will not consider the pandapower \"ext_grid\" as there "
Traceback (most recent call last):
  File "/home/boguslawskieva/env_with_setpoint_new_implementation/test.py", line 200, in <module>
    copy.deepcopy(env_gym)
  File "/usr/lib/python3.12/copy.py", line 162, in deepcopy
    y = _reconstruct(x, memo, *rv)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 259, in _reconstruct
    state = deepcopy(state, memo)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 136, in deepcopy
    y = copier(x, memo)
        ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 221, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
                             ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 162, in deepcopy
    y = _reconstruct(x, memo, *rv)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 259, in _reconstruct
    state = deepcopy(state, memo)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 136, in deepcopy
    y = copier(x, memo)
        ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 221, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
                             ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 162, in deepcopy
    y = _reconstruct(x, memo, *rv)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 259, in _reconstruct
    state = deepcopy(state, memo)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 136, in deepcopy
    y = copier(x, memo)
        ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 221, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
                             ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/copy.py", line 143, in deepcopy
    y = copier(memo)
        ^^^^^^^^^^^^
  File "/data/boguslawskieva/virtual_environments/venv_multi_agent/lib/python3.12/site-packages/lightsim2grid/lightSimBackend.py", line 1418, in __deepcopy__
    result = self.copy()
             ^^^^^^^^^^^
  File "/data/boguslawskieva/virtual_environments/venv_multi_agent/lib/python3.12/site-packages/lightsim2grid/lightSimBackend.py", line 1531, in copy
    res._grid = mygrid.copy()
                ^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'copy'

[BDonnot]

Hello,

I thinks it's a lightsim2grid issue. Can you put an issue there too please?
Thanks 😊

Also you should probably avoid doing deep copies of grid2op environment. It does not really work well. The best is probably to use the copy function that does the trick most of the time.

If you really want to rely on copy.deepcopy you can
Implementi the ___deepcopy___ operator (exact name needs to be checked) for GymEnv and make sure to use the grid2op copy() and not deep copy there

[EBoguslawski]

Hello,

Thank you for your answer 😃

• Noted, however I think I don't really have a choice. In the l2rpn_baselines.PPO_SB3.utils.SB3Agent class, a copy.deepcopy is performed on the nn_kwargs argument during initialization, the latter containing a GymEnv in its keys / values (needed to train the agent).
• I didn't understand, did you mean "GymEnv environment" instead of "grid2op environment"? The error appears if I try to deep copy a GymEnv env not a grid2op env.
• I cannot find a copy method for the GymEnv class

Alright, I will also put the issue on LightSim2Grid

[EBoguslawski]

Hello again,

I think I have managed to do what you suggested and I would like your advice before doing the PR.

The solution would be to add a __deepcopy__ method to GymEnv but to avoid touching my library I just created a child class GymEnvWithDeepCopy for the exemple.
I didn't manage to make the copy.deepcopy(self, memo) idea work. It raised a RecursionError: maximum recursion depth exceeded error, and I didn't find any solution.
But here is another proposition:

import grid2op
from grid2op.gym_compat import GymEnv
from lightsim2grid import LightSimBackend
import copy


# I add a __deepcopy__ method to a children class from GymEnv
class GymEnvWithDeepCopy(GymEnv):
     
    def __init__(self, env_init, shuffle_chronics = True, render_mode = "rgb_array", with_forecast = False):
          super().__init__(env_init, shuffle_chronics, render_mode, with_forecast)

    def __deepcopy__(self, memo=None):

        if memo is None:
            memo = {}
        
        # I initialize another instance
        res = type(self)(
            self.init_env.copy(),
            shuffle_chronics=self._shuffle_chronics,
            render_mode=self.init_env.render_mode,
            with_forecast=self.init_env.with_forecast
        )
        
        ##########################
        # I deepcopy the other attributes
        for key, value in vars(self).items():
                if key not in ('init_env', '_shuffle_chronics'):  # Skip already copied attributes
                    setattr(res, key, copy.deepcopy(value, memo))
        ##########################
        
        return res

# I create a GymEnvWithDeepCopy env from a grid2op env with a LightSimBackend
env = grid2op.make("l2rpn_case14_sandbox", backend=LightSimBackend())
env_gym_withDP = GymEnvWithDeepCopy(env)

# This line was raising an error when using GymEnv
env_gym_withDP_copy = copy.deepcopy(env_gym_withDP)

print("Do they have the same attributes' names?")
print("Yes" if dir(env_gym_withDP) == dir(env_gym_withDP_copy) else "No")
# I obtain Yes which means that every attributes has been initialized or deepcopied

I think the code between ### is useful because even if the other attributes (action_space, observation_space, reward_range, metadata) seem to be initialized in __init__ from init_env and not modified after, the user might impose subsequent modifications.

Current (and wanted) output:

Do they have the same attributes' names?
Yes

What do you think?