Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(load): add arrow endpoints #2200

Open
wants to merge 43 commits into
base: dev
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
ad63883
feat: first commit
TheoPascoli Oct 24, 2024
62f1b1d
feat: first commit
TheoPascoli Oct 24, 2024
cc01641
Merge branch 'dev' into feat/add-load-endpoints-with-arrow
TheoPascoli Oct 24, 2024
d73c8de
Merge branch 'dev' into feat/add-load-endpoints-with-arrow
TheoPascoli Oct 25, 2024
998c04e
feat(bc): use `update_config` instead of `update_bc` for multiple upd…
MartinBelthle Oct 25, 2024
f4206fe
build(python): bump python version to use v3.11 (#2164)
MartinBelthle Oct 29, 2024
f8b0f8a
feat(ts-gen): display progress bar via websockets (#2194)
MartinBelthle Oct 29, 2024
ebd2df4
feat: first commit
TheoPascoli Oct 24, 2024
9a3591a
feat: first commit
TheoPascoli Oct 24, 2024
fddf3b8
feat: first commit
TheoPascoli Oct 24, 2024
582aed0
feat: first commit
TheoPascoli Oct 24, 2024
72ec467
Merge branch 'feat/add-load-endpoints-with-arrow' of https://github.c…
TheoPascoli Oct 30, 2024
43a5e40
feat: Refactor load management to use LoadInfoDTO
TheoPascoli Oct 30, 2024
4e74852
feat: refactor load series API response model
TheoPascoli Oct 30, 2024
c99cdb1
feat: add support for pyarrow in load management
TheoPascoli Oct 30, 2024
0a4e6f5
feat: refactor load series api response model
TheoPascoli Oct 30, 2024
1b6dc2b
feat: add support for pyarrow in load management
TheoPascoli Oct 30, 2024
cae2930
Merge remote-tracking branch 'origin/feat/add-load-endpoints-with-arr…
TheoPascoli Oct 30, 2024
1b86d9a
feat: fix whitespace issue in load management module
TheoPascoli Oct 30, 2024
f45f065
feat: refactor load management response
TheoPascoli Oct 31, 2024
6506459
Merge remote-tracking branch 'origin/dev' into feat/add-load-endpoint…
TheoPascoli Oct 31, 2024
213cbaf
feat: refactor `get_load_matrix` method to improve readability
TheoPascoli Nov 4, 2024
7b5e0a0
feat: add integration test for load series endpoint
TheoPascoli Nov 4, 2024
37b7108
feat: add endpoint to update load series data
TheoPascoli Nov 6, 2024
1ef1ff1
feat: refactor LoadDTO and add LoadProperties model
TheoPascoli Nov 6, 2024
4a234e6
feat: refactor link path handling in link management.
TheoPascoli Nov 6, 2024
c0a82de
Merge remote-tracking branch 'origin/dev' into feat/add-load-endpoint…
TheoPascoli Nov 7, 2024
e91dc71
Merge remote-tracking branch 'origin/dev' into feat/add-load-endpoint…
TheoPascoli Nov 7, 2024
997803a
feat: refactor load series endpoint paths and add data conversion
TheoPascoli Nov 7, 2024
255b104
Merge branch 'dev' into feat/add-load-endpoints-with-arrow
TheoPascoli Nov 12, 2024
aff9c83
feat: remove JSON format support for load matrix endpoints
TheoPascoli Nov 12, 2024
d63cb8e
feat: add docstrings and fix IO imports in load management
TheoPascoli Nov 12, 2024
a522a22
feat: add docstrings and fix IO imports in load management
TheoPascoli Nov 12, 2024
8fe2863
Merge remote-tracking branch 'origin/dev' into feat/add-load-endpoint…
TheoPascoli Nov 12, 2024
9d55b36
Merge remote-tracking branch 'origin/feat/add-load-endpoints-with-arr…
TheoPascoli Nov 12, 2024
5df3960
feat: add docstrings and fix IO imports in load management
TheoPascoli Nov 12, 2024
09bc218
Merge branch 'dev' into feat/add-load-endpoints-with-arrow
TheoPascoli Nov 12, 2024
7e3b03a
feat: remove web considerations that were inside the business layer
TheoPascoli Nov 28, 2024
6277ae0
Merge remote-tracking branch 'origin/dev' into feat/add-load-endpoint…
TheoPascoli Dec 3, 2024
5fd6880
feat: set up GET endpoint that returns a arrow matrix with MatrixInde…
TheoPascoli Dec 5, 2024
feb957f
feat: remove useless load_model.py
TheoPascoli Dec 5, 2024
cb3d3f4
feat: remove useless load_model.py
TheoPascoli Dec 5, 2024
fe6acbd
feat: change way of dealing with the writing of the feather file
TheoPascoli Dec 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
feat: first commit
TheoPascoli committed Oct 30, 2024
commit 9a3591ab805721bff95cac76d072cc0448ece002
27 changes: 27 additions & 0 deletions antarest/study/business/load_management.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Copyright (c) 2024, RTE (https://www.rte-france.com)
#
# See AUTHORS.txt
#
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
#
# SPDX-License-Identifier: MPL-2.0
#
# This file is part of the Antares project.
import typing as t

from antarest.study.model import MatrixFormat, Study
from antarest.study.storage.storage_service import StudyStorageService


class LoadOutput:
pass


class LoadManager:
def __init__(self, storage_service: StudyStorageService) -> None:
self.storage_service = storage_service

def get_load_matrix(self, study: Study, area_id: str, matrix_format: t.Optional[MatrixFormat]) -> LoadOutput:
return LoadOutput()
5 changes: 5 additions & 0 deletions antarest/study/model.py
Original file line number Diff line number Diff line change
@@ -506,6 +506,11 @@ def suffix(self) -> str:
return mapping[self]


class MatrixFormat(str, enum.Enum): # Todo set new StrEnum when we upgrade to python 3.11
JSON = "json"
ARROW = "arrow"


class StudyDownloadDTO(AntaresBaseModel):
"""
DTO used to download outputs
2 changes: 2 additions & 0 deletions antarest/study/service.py
Original file line number Diff line number Diff line change
@@ -86,6 +86,7 @@
from antarest.study.business.district_manager import DistrictManager
from antarest.study.business.general_management import GeneralManager
from antarest.study.business.link_management import LinkInfoDTO, LinkManager
from antarest.study.business.load_management import LoadManager
from antarest.study.business.matrix_management import MatrixManager, MatrixManagerError
from antarest.study.business.optimization_management import OptimizationManager
from antarest.study.business.playlist_management import PlaylistManager
@@ -365,6 +366,7 @@ def __init__(
self.adequacy_patch_manager = AdequacyPatchManager(self.storage_service)
self.advanced_parameters_manager = AdvancedParamsManager(self.storage_service)
self.hydro_manager = HydroManager(self.storage_service)
self.load_manager = LoadManager(self.storage_service)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally speaking for the other matrices, what is your vision here ? For hydro for instance, we want to add the code inside the HydroManager or do we want to create a specific manager on the side ? I personally prefer the 1st option

self.allocation_manager = AllocationManager(self.storage_service)
self.properties_manager = PropertiesManager(self.storage_service)
self.renewable_manager = RenewableManager(self.storage_service)
23 changes: 22 additions & 1 deletion antarest/study/web/study_data_blueprint.py
Original file line number Diff line number Diff line change
@@ -69,13 +69,14 @@
from antarest.study.business.district_manager import DistrictCreationDTO, DistrictInfoDTO, DistrictUpdateDTO
from antarest.study.business.general_management import GeneralFormFields
from antarest.study.business.link_management import LinkInfoDTO
from antarest.study.business.load_management import LoadOutput
from antarest.study.business.optimization_management import OptimizationFormFields
from antarest.study.business.playlist_management import PlaylistColumns
from antarest.study.business.scenario_builder_management import Rulesets, ScenarioType
from antarest.study.business.table_mode_management import TableDataDTO, TableModeType
from antarest.study.business.thematic_trimming_field_infos import ThematicTrimmingFormFields
from antarest.study.business.timeseries_config_management import TSFormFields
from antarest.study.model import PatchArea, PatchCluster
from antarest.study.model import MatrixFormat, PatchArea, PatchCluster
from antarest.study.service import StudyService
from antarest.study.storage.rawstudy.model.filesystem.config.binding_constraint import (
BindingConstraintFrequency,
@@ -524,6 +525,26 @@ def update_inflow_structure(
study = study_service.check_study_access(uuid, StudyPermissionType.WRITE, params)
study_service.hydro_manager.update_inflow_structure(study, area_id, values)

@bp.get(
"/{uuid}/{area_id}/load/series",
TheoPascoli marked this conversation as resolved.
Show resolved Hide resolved
tags=[APITag.study_data],
summary="Get load series data",
response_model=LoadOutput,
)
def get_load_series(
uuid: str,
area_id: str,
matrix_format: t.Optional[MatrixFormat] = None,
current_user: JWTUser = Depends(auth.get_current_user),
) -> LoadOutput:
logger.info(
msg=f"Getting load series data for area {area_id} of study {uuid}",
extra={"user": current_user.id},
)
params = RequestParameters(user=current_user)
study = study_service.check_study_access(uuid, StudyPermissionType.READ, params)
return study_service.load_manager.get_load_matrix(study, area_id, matrix_format)

@bp.put(
"/studies/{uuid}/matrix",
tags=[APITag.study_data],