Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

synch with dev 1.11.0 #671

Merged
merged 44 commits into from
Dec 3, 2024
Merged
Changes from 1 commit
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
79a4483
fix issue grid2op#657
BDonnot Nov 6, 2024
c30ac8a
fix some error after previous commit
BDonnot Nov 6, 2024
edc1565
fix some error after previous commits
BDonnot Nov 7, 2024
9d02624
fixing bugs
BDonnot Nov 7, 2024
bd1679d
Merge pull request #658 from BDonnot/bd_dev
BDonnot Nov 7, 2024
80a54f9
adding the 'caching' kwargs to FromMultiEpisodeData
BDonnot Nov 7, 2024
914e003
fixing some bugs after changing the get_id / set_id of MultiEpisodeData
BDonnot Nov 7, 2024
c94c1da
now failing if the init state options raises a warning
BDonnot Nov 7, 2024
5c14343
now failing if the init state options raises a warning
BDonnot Nov 7, 2024
f259521
fix broken tests
BDonnot Nov 8, 2024
b658793
improve reading speed of FromEpisodeData by not reading everything, s…
BDonnot Nov 8, 2024
ccdc626
some improvment for base agents
BDonnot Nov 8, 2024
7acf066
fix an non issue spotted by sonarcube [skip ci]
BDonnot Nov 8, 2024
14e0bdb
Merge pull request #660 from BDonnot/bd_dev
BDonnot Nov 8, 2024
bbec434
fixing an issue in the action
BDonnot Nov 19, 2024
2b6a03e
Merge pull request #662 from BDonnot/bd_dev
BDonnot Nov 19, 2024
253be86
some fixes, need proper tests now
BDonnot Nov 19, 2024
4ac67de
fix typo in Kirchhoff name and fix issue with init topology in foreca…
BDonnot Nov 20, 2024
8407f6d
Merge pull request #663 from BDonnot/bd_dev
BDonnot Nov 20, 2024
ad607b9
by default grid2op env will not take the name of the backend class
BDonnot Nov 20, 2024
47ae6b2
fixing the automatic class with the new name change + improve automat…
BDonnot Nov 21, 2024
964899f
fix broken observation tests
BDonnot Nov 21, 2024
22342e2
fix some broken tests
BDonnot Nov 21, 2024
6637d2d
in the middle of fixes [skip ci]
BDonnot Nov 21, 2024
c731a06
fixing bugs in CI
BDonnot Nov 22, 2024
e03f5d7
small refacto for sonarcube
BDonnot Nov 22, 2024
d0eacac
fix broken tests
BDonnot Nov 22, 2024
37029ad
fix a bug (variable in error message not initialized)
BDonnot Nov 22, 2024
26e8e33
Merge pull request #664 from BDonnot/bd_dev
BDonnot Nov 22, 2024
1713fca
fix issue grid2op#665
BDonnot Nov 28, 2024
c2db05a
use manylinux_2_28 instead of manylinux2014 which is now EOL
BDonnot Nov 28, 2024
2483987
use manylinux_2_28 instead of manylinux2014 which is now EOL
BDonnot Nov 28, 2024
c6754d1
Merge pull request #668 from BDonnot/bd_dev
BDonnot Nov 28, 2024
85d2e3e
fix issue #667
BDonnot Nov 28, 2024
e83face
fix issues spotted in the CI after the act.as_serializable_dict() fix…
BDonnot Nov 29, 2024
627f52d
fix issues spotted in the CI after the act.as_serializable_dict() fix…
BDonnot Nov 29, 2024
abddf01
Fix bug in the MultifolderWithCache.seed method and add a test
EBoguslawski Nov 29, 2024
8dfe10c
fix typing error
EBoguslawski Nov 29, 2024
6e59656
remove useless code
EBoguslawski Nov 29, 2024
5a335c0
Merge pull request #670 from EBoguslawski/dev_1.11.0_seed_chronics
BDonnot Dec 1, 2024
f10836e
fix issue when init an env from handlers
BDonnot Dec 2, 2024
ea78fdb
fix a bug when setting thermal limit with obs_env
BDonnot Dec 2, 2024
f462820
fix a bug introduced in the previous fix
BDonnot Dec 2, 2024
f282298
Merge pull request #669 from BDonnot/bd_dev
BDonnot Dec 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
adding the 'caching' kwargs to FromMultiEpisodeData
Signed-off-by: DONNOT Benjamin <benjamin.donnot@rte-france.com>
  • Loading branch information
BDonnot committed Nov 8, 2024
commit 80a54f91c139b782ef0286e7453a07427689761b
4 changes: 4 additions & 0 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
@@ -99,6 +99,8 @@ Native multi agents support:

[1.11.0] - 202x-yy-zz
-----------------------
- [BREAKING] Change for `FromMultiEpisodeData` that disables the caching by default
when creating the data.
- [FIXED] issue https://github.com/Grid2op/grid2op/issues/657
- [FIXED] missing an import on the `MaskedEnvironment` class
- [ADDED] possibility to set the "thermal limits" when calling `env.reset(..., options={"thermal limit": xxx})`
@@ -113,6 +115,8 @@ Native multi agents support:
"chronics_hander" in the ObsEnv behaves (it now fully implements the public interface of
a "real" chronic_handler)
- [IMPROVED] error message in the `FromNPY` class when the backend is checked
- [IMRPOVED] the `FromMultiEpisodeData` class with the addition of the `caching`
kwargs to allow / disable caching (which was default behavior in previous version)

[1.10.4] - 2024-10-15
-------------------------
72 changes: 52 additions & 20 deletions grid2op/Chronics/fromMultiEpisodeData.py
Original file line number Diff line number Diff line change
@@ -7,16 +7,9 @@
# This file is part of Grid2Op, Grid2Op a testbed platform to model sequential decision making in power systems.

from datetime import datetime, timedelta
import os
import numpy as np
import copy
import warnings
from typing import Optional, Union, List, Dict, Literal
from pathlib import Path

from grid2op.Exceptions import (
ChronicsError, ChronicsNotFoundError
)
from grid2op.Exceptions import ChronicsError

from grid2op.Chronics.gridValue import GridValue

@@ -40,6 +33,17 @@ class FromMultiEpisodeData(GridValue):
- to make sure you are running the exact same episode, you need to create the environment
with the :class:`grid2op.Opponent.FromEpisodeDataOpponent` opponent

.. versionchanged:: 1.11.0
Before versin 1.11.0 this class would load all the data in memory at the creation of the environment,
which could take lots of time and memory but once done a call to `env.reset` would be really fast.

From grid2op >= 1.11.0 a kwargs `caching` has been added (default value is ``FALSE``) which
does not load everything in memory which makes it more memory efficient and (maybe) more time saving
(if some data happened to be loaded but never used). The default behaviour has then
changed.

You can still benefit from previous behaviour by loading with `caching=True`

Examples
---------
You can use this class this way:
@@ -110,21 +114,39 @@ def __init__(self,
max_iter=-1,
start_datetime=datetime(year=2019, month=1, day=1),
chunk_size=None,
list_perfect_forecasts=None, # TODO
list_perfect_forecasts=None,
caching : bool=False,
**kwargs, # unused
):
super().__init__(time_interval, max_iter, start_datetime, chunk_size)
self.li_ep_data = [FromOneEpisodeData(path,
ep_data=el,
time_interval=time_interval,
max_iter=max_iter,
chunk_size=chunk_size,
list_perfect_forecasts=list_perfect_forecasts,
start_datetime=start_datetime)
for el in li_ep_data
]
self._caching : bool = bool(caching)
self._path = path
self._chunk_size = chunk_size
self._list_perfect_forecasts = list_perfect_forecasts
if self._caching:
self.li_ep_data = [FromOneEpisodeData(path,
ep_data=el,
time_interval=time_interval,
max_iter=max_iter,
chunk_size=chunk_size,
list_perfect_forecasts=list_perfect_forecasts,
start_datetime=start_datetime)
for el in li_ep_data
]
self._input_li_ep_data = None
else:
self.li_ep_data = [None for el in li_ep_data]
self._input_li_ep_data = li_ep_data
self._prev_cache_id = len(self.li_ep_data) - 1
self.data = self.li_ep_data[self._prev_cache_id]
if self.data is None:
self.data = FromOneEpisodeData(self._path,
ep_data=self._input_li_ep_data[self._prev_cache_id],
time_interval=self.time_interval,
max_iter=self.max_iter,
chunk_size=self._chunk_size,
list_perfect_forecasts=self._list_perfect_forecasts,
start_datetime=self.start_datetime)
self._episode_data = self.data._episode_data # used by the fromEpisodeDataOpponent

def next_chronics(self):
@@ -144,6 +166,15 @@ def initialize(
):

self.data = self.li_ep_data[self._prev_cache_id]
if self.data is None:
# data was not in cache:
self.data = FromOneEpisodeData(self._path,
ep_data=self._input_li_ep_data[self._prev_cache_id],
time_interval=self.time_interval,
max_iter=self.max_iter,
chunk_size=self._chunk_size,
list_perfect_forecasts=self._list_perfect_forecasts,
start_datetime=self.start_datetime)
self.data.initialize(
order_backend_loads,
order_backend_prods,
@@ -168,7 +199,8 @@ def check_validity(self, backend):
def forecasts(self):
return self.data.forecasts()

def tell_id(self, id_num, previous=False):
def tell_id(self, id_num: str, previous=False):
path_, id_num = id_num.split("@")
id_num = int(id_num)
if not isinstance(id_num, (int, dt_int)):
raise ChronicsError("FromMultiEpisodeData can only be used with `tell_id` being an integer "
@@ -182,7 +214,7 @@ def tell_id(self, id_num, previous=False):
self._prev_cache_id %= len(self.li_ep_data)

def get_id(self) -> str:
return f'{self._prev_cache_id }'
return f'{self._path}@{self._prev_cache_id}'

def max_timestep(self):
return self.data.max_timestep()
16 changes: 12 additions & 4 deletions grid2op/tests/test_env_from_episode.py
Original file line number Diff line number Diff line change
@@ -407,7 +407,7 @@ def test_given_example_multiepdata(self):
env2 = grid2op.make(env_name,
test=True,
chronics_class=FromMultiEpisodeData,
data_feeding_kwargs={"li_ep_data": li_episode},
data_feeding_kwargs={"li_ep_data": li_episode, "caching": True},
opponent_class=FromEpisodeDataOpponent,
opponent_attack_cooldown=1,
_add_to_name=type(self).__name__,
@@ -551,7 +551,10 @@ def setUp(self) -> None:
def tearDown(self) -> None:
self.env.close()
return super().tearDown()


def do_i_cache(self):
return False

def test_basic(self):
"""test injection, without opponent nor maintenance"""
obs = self.env.reset()
@@ -565,7 +568,7 @@ def test_basic(self):
env = grid2op.make(self.env_name,
test=True,
chronics_class=FromMultiEpisodeData,
data_feeding_kwargs={"li_ep_data": ep_data},
data_feeding_kwargs={"li_ep_data": ep_data, "caching": self.do_i_cache()},
opponent_attack_cooldown=99999999,
opponent_attack_duration=0,
opponent_budget_per_ts=0.,
@@ -607,6 +610,11 @@ def test_basic(self):
obs, reward, done, info = env.step(env.action_space())
assert env.chronics_handler.get_id() == "1"



class TestTSFromMultieEpisodeWithCache(TestTSFromMultieEpisode):
def do_i_cache(self):
return True


if __name__ == "__main__":
unittest.main()