Multiple Maximum Generation and Pumping for Hydro (Refactoring)

.github/workflows/windows-vcpkg.yml

@@ -85,11 +85,11 @@ jobs:
         ortools-url: ${{env.ORTOOLS_URL}}
         ortools-dir: ${{env.ORTOOLS_DIR}}
 
-    - name: Setup Python 3.11
+    - name: Setup Python 3.12
       uses: actions/setup-python@v4
       with:
         architecture: 'x64'
-        python-version: '3.11'
+        python-version: '3.12'
 
     - name: Install pip dependencies if necessary
       run: pip install -r src/tests/examples/requirements.txt

docs/reference-guide/03-commands.md

@@ -178,7 +178,7 @@ this command allows to state, for each kind of time-series, whether it should be
 the available set (be it ready-made or Antares-generated) _**OR**_ should take a user-defined value
 (in the former case, the default "rand" value should be kept; in the latter, the value should be the reference number of the time-series to use). Multiple simulation profiles can be defined and archived. The default active profile gives the "rand" status for all time-series in all areas (full probabilistic simulation).
 
-  Regarding Hydro time-series, the scenario builder gives, in addition to the assignment of a specific number to use for the inflows time-series, the ability to define the initial reservoir level to use for each MC year, also hydro power credits scenario builder is available to support time-series for Maximum Generation and Maximum Pumping because the number of TS's for ROR, Hydro Storage and Minimum Generation can be different than the  number of TS's for Maximum Generation and Maximum Pumping.
+  Regarding Hydro time-series, the scenario builder gives, in addition to the assignment of a specific number to use for the inflows time-series, the ability to define the initial reservoir level to use for each MC year, also hydro max power scenario builder is available to support time-series for Maximum Generation and Maximum Pumping because the number of TS's for ROR, Hydro Storage and Minimum Generation can be different than the  number of TS's for Maximum Generation and Maximum Pumping.
 
 - **MC Scenario playlist** For each Monte-Carlo year of the simulation defined in the "Simulation" active window,
 this command allows to state whether a MC year prepared for the simulation should be actually simulated or not.

docs/reference-guide/04-active_windows.md

@@ -97,10 +97,10 @@ These two parts are detailed hereafter.
 
 ### RIGHT PART: Time-series management
 
-For the different kinds of time-series that Antares manages in a non-deterministic way (load, thermal generation, hydro power, hydro power credits, wind power, solar power or renewable depending on the option chosen):
+For the different kinds of time-series that Antares manages in a non-deterministic way (load, thermal generation, hydro power, hydro max power, wind power, solar power or renewable depending on the option chosen):
 
 1. **Choice of the kind of time-series to use**
-Either « ready-made » or «stochastic » (i.e. Antares-generated), defined by setting the value to either "on" or "off". Exception is hydro power credits that can only be « ready-made ». Note that for Thermal time-series, the cluster-wise parameter may overrule this global parameter (see Thermal window description below).
+Either « ready-made » or «stochastic » (i.e. Antares-generated), defined by setting the value to either "on" or "off". Exception is hydro max power that can only be « ready-made ». Note that for Thermal time-series, the cluster-wise parameter may overrule this global parameter (see Thermal window description below).
 
 2. **For stochastic TS only**:
     - **Number** Number of TS to generate

docs/reference-guide/13-file-format.md

@@ -1,11 +1,11 @@
 # Study format changes
 This is a list of all recent changes that came with new Antares Simulator features. The main goal of this document is to lower the costs of changing existing interfaces, both GUI and scripts.
 ## v8.8.0
-For each area, new files are added **input/hydro/series/<area>/maxgen.txt** and **input/hydro/series/<area>/maxpump.txt**. These files have one or more columns, and 8760 rows. The number of columns in these two files must be the same, if there is more than one column in each file, but if there is just one column for example in maxgen.txt file, maxpump.txt file can have more than one column and vice versa. For old studies, file **input/hydro/common/capacity/maxpower_<area>** will be deleted after upgrading the study and corresponding data from that file will be copied to already mentioned new files, even if study is upgraded or not. In that case maxgen.txt and maxpump.txt files will have just one column and 8760 rows.
+For each area, new files are added **input/hydro/series/<area>/maxHourlyGenPower.txt** and **input/hydro/series/<area>/maxHourlyPumpPower.txt**. These files have one or more columns, and 8760 rows. The number of columns in these two files must be the same, if there is more than one column in each file, but if there is just one column for example in maxHourlyGenPower.txt file, maxHourlyPumpPower.txt file can have more than one column and vice versa. For old studies, file **input/hydro/common/capacity/maxpower_<area>** will be deleted after upgrading the study and corresponding data from that file will be copied to already mentioned new files, even if study is upgraded or not. In that case maxHourlyGenPower.txt and maxHourlyPumpPower.txt files will have just one column and 8760 rows.
 
-Also for each area, new files are added **input/hydro/common/capacity/maxhoursGen_<area>** and **input/hydro/common/capacity/maxhoursPump_<area>**. These files have just one column and 365 rows. For old studies, file **input/hydro/common/capacity/maxpower_<area>** will be deleted after upgrading the study and corresponding data from that file will be copied to already mentioned new files, even if study is upgraded or not.
+Also for each area, new files are added **input/hydro/common/capacity/maxDailyGenEnergy_<area>** and **input/hydro/common/capacity/maxDailyPumpEnergy_<area>**. These files have just one column and 365 rows. For old studies, file **input/hydro/common/capacity/maxpower_<area>** will be deleted after upgrading the study and corresponding data from that file will be copied to already mentioned new files, even if study is upgraded or not.
 ### Input
-Under `Configure/MC Scenario Builder` new section added `Hydro-Power-Credits`
+Under `Configure/MC Scenario Builder` new section added `Hydro Max Power`
 In the existing file **settings/scenariobuilder.dat**, under **<ruleset>** section following properties added:
 * **hgp,<area>,<year> = <hgp-value>**
 ## v8.7.0

src/libs/antares/constants.h

@@ -42,6 +42,8 @@
 /*! Hours per year */
 #define HOURS_PER_YEAR 8760
 
+#define HOURS_PER_DAY 24U
+
 namespace Antares::Constants
 {
 extern const std::array<unsigned int, 12> daysPerMonth;

src/libs/antares/study/CMakeLists.txt

@@ -23,8 +23,8 @@ set(SRC_STUDY_SCENARIO_BUILDER
         scenario-builder/ThermalTSNumberData.h
         scenario-builder/HydroTSNumberData.h
         scenario-builder/HydroTSNumberData.cpp
-        scenario-builder/HydroPowerCreditsTSNumberData.h
-        scenario-builder/HydroPowerCreditsTSNumberData.cpp
+        scenario-builder/HydroMaxPowerTSNumberData.h
+        scenario-builder/HydroMaxPowerTSNumberData.cpp
         scenario-builder/SolarTSNumberData.cpp
         scenario-builder/solarTSNumberData.h
         scenario-builder/WindTSNumberData.h
@@ -137,8 +137,10 @@ set(SRC_STUDY_PART_HYDRO
         parts/hydro/allocation.h
         parts/hydro/allocation.hxx
         parts/hydro/allocation.cpp
-        parts/hydro/datatransfer.h
-        parts/hydro/datatransfer.cpp
+        parts/hydro/hydromaxtimeseriesreader.h
+        parts/hydro/hydromaxtimeseriesreader.cpp
+        parts/hydro/pair-of-integers.h
+        parts/hydro/pair-of-integers.cpp
 )
 source_group("study\\part\\hydro" FILES ${SRC_STUDY_PART_HYDRO})
 

src/libs/antares/study/area/area.cpp

@@ -280,7 +280,7 @@ void Area::resizeAllTimeseriesNumbers(uint n)
         solar.series->timeseriesNumbers.clear();
         wind.series->timeseriesNumbers.clear();
         hydro.series->timeseriesNumbers.clear();
-        hydro.series->timeseriesNumbersPowerCredits.clear();
+        hydro.series->timeseriesNumbersHydroMaxPower.clear();
         for (auto& namedLink : links)
         {
             AreaLink* link = namedLink.second;
@@ -293,7 +293,7 @@ void Area::resizeAllTimeseriesNumbers(uint n)
         solar.series->timeseriesNumbers.resize(1, n);
         wind.series->timeseriesNumbers.resize(1, n);
         hydro.series->timeseriesNumbers.resize(1, n);
-        hydro.series->timeseriesNumbersPowerCredits.resize(1, n);
+        hydro.series->timeseriesNumbersHydroMaxPower.resize(1, n);
         for (auto& namedLink : links)
         {
             AreaLink* link = namedLink.second;

src/libs/antares/study/area/list.cpp

@@ -901,34 +901,47 @@ static bool AreaListLoadFromFolderSingleArea(Study& study,
             buffer.clear() << study.folderInput << SEP << "hydro" << SEP << "prepro";
             ret = area.hydro.prepro->loadFromFolder(study, area.id, buffer.c_str()) && ret;
         }
-        if (area.hydro.series && (!options.loadOnlyNeeded || !area.hydro.prepro)) // Series
-        {
-            buffer.clear() << study.folderInput << SEP << "hydro" << SEP << "series";
-            ret = area.hydro.series->loadFromFolder(study, area.id, buffer) && ret;
-        }
 
-        if (area.hydro.series && study.header.version < 870)
+        if (auto* hydroSeries = area.hydro.series; hydroSeries)
         {
-            std::shared_ptr<DataTransfer> datatransfer = std::make_shared<DataTransfer>();
-            buffer.clear() << study.folderInput << SEP << "hydro";
+            if (!options.loadOnlyNeeded || !area.hydro.prepro) // Series
+            {
+                buffer.clear() << study.folderInput << SEP << "hydro" << SEP << "series";
+                ret = hydroSeries->loadGenerationTS(area.id, buffer, study.header.version) && ret;
 
-            datatransfer->LoadFromFolder(study, buffer, area);
-            datatransfer->SupportForOldStudies(study, buffer, area);
-        }
+                hydroSeries->EqualizeGenerationTSsizes(
+                  area, study.usedByTheSolver, study.gotFatalError);
+            }
 
-        if (area.hydro.series)
-        {
-            buffer.clear() << study.folderInput << SEP << "hydro" << SEP << "series";
-            ret = area.hydro.series->LoadHydroPowerCredits(study, area.id, buffer) && ret;
+            if (study.header.version < 870)
+            {
+                buffer.clear() << study.folderInput << SEP << "hydro";
+
+                HydroMaxTimeSeriesReader reader;
+                ret = reader(buffer, area, study.usedByTheSolver) && ret;
+            }
+
+            if (study.header.version >= 870)
+            {
+                buffer.clear() << study.folderInput << SEP << "hydro" << SEP << "series";
+                ret = hydroSeries->LoadMaxPower(area.id, buffer) && ret;
+
+                if (study.usedByTheSolver)
+                {
+                    hydroSeries->EqualizeMaxPowerTSsizes(area, study.gotFatalError);
+                }
+                else
+                    hydroSeries->setHydroModulability(area);
+            }
+
+            hydroSeries->resizeTSinDeratedMode(
+              study.parameters.derated, study.header.version, study.usedByTheSolver);
         }
 
         buffer.clear() << study.folderInput << SEP << "hydro" << SEP << "common" << SEP
-                       << "capacity" << SEP << "maxpower_" << area.id << '.'
-                       << study.inputExtension;
-
-        bool exists = IO::File::Exists(buffer);
+                       << "capacity" << SEP << "maxpower_" << area.id << ".txt";
 
-        if (study.header.version >= 870 && exists)
+        if (bool exists = IO::File::Exists(buffer); study.header.version >= 870 && exists)
         {
             IO::File::Delete(buffer);
         }
@@ -1636,13 +1649,7 @@ void AreaList::removeLoadTimeseries()
 void AreaList::removeHydroTimeseries()
 {
     each([&](Data::Area& area) {
-        area.hydro.series->ror.reset(1, HOURS_PER_YEAR);
-        area.hydro.series->storage.reset(1, DAYS_PER_YEAR);
-        area.hydro.series->mingen.reset(1, HOURS_PER_YEAR);
-        area.hydro.series->maxgen.reset(1, HOURS_PER_YEAR);
-        area.hydro.series->maxpump.reset(1, HOURS_PER_YEAR);
-        area.hydro.series->count = 1;
-        area.hydro.series->countpowercredits = 1;
+        area.hydro.series->reset();
     });
 }
 

src/libs/antares/study/area/scratchpad.cpp

@@ -32,9 +32,7 @@
 
 using namespace Yuni;
 
-namespace Antares
-{
-namespace Data
+namespace Antares::Data
 {
 AreaScratchpad::TimeseriesData::TimeseriesData(Area& area) :
         load(area.load.series->timeSeries), solar(area.solar.series->timeSeries), wind(area.wind.series->timeSeries)
@@ -56,7 +54,7 @@ AreaScratchpad::AreaScratchpad(const StudyRuntimeInfos& rinfos, Area& area) : ts
         originalMustrunSum[h] = std::numeric_limits<double>::quiet_NaN();
     }
 
-     // Fatal hors hydro
+    // Fatal hors hydro
     {
         double sum;
         uint w;
@@ -77,6 +75,18 @@ AreaScratchpad::AreaScratchpad(const StudyRuntimeInfos& rinfos, Area& area) : ts
         }
     }
 
+    //  Hourly maximum generation/pumping power matrices and their number of TS's (width of matrices)
+    auto const& maxHourlyGenPower = area.hydro.series->maxHourlyGenPower;
+    auto const& maxHourlyPumpPower = area.hydro.series->maxHourlyPumpPower;
+    uint nbOfMaxPowerTimeSeries = area.hydro.series->maxPowerTScount();
+
+    //  Setting width and height of daily mean maximum generation/pumping power matrices
+    meanMaxDailyGenPower.reset(nbOfMaxPowerTimeSeries, DAYS_PER_YEAR);
+    meanMaxDailyPumpPower.reset(nbOfMaxPowerTimeSeries, DAYS_PER_YEAR);
+
+    // Instantiate daily mean maximum generation/pumping power matrices
+    CalculateMeanDailyMaxPowerMatrices(maxHourlyGenPower, maxHourlyPumpPower, nbOfMaxPowerTimeSeries);
+
     // ===============
     // hydroHasMod
     // ===============
@@ -85,13 +95,12 @@ AreaScratchpad::AreaScratchpad(const StudyRuntimeInfos& rinfos, Area& area) : ts
     // Hydro generation permission
     // ------------------------------
     // Useful whether we use a heuristic target or not
-     bool hydroGenerationPermission = false;
+    bool hydroGenerationPermission = false;
 
-    // ... Getting hydro max power
-    auto const& maxPower = area.hydro.series->maxgen;
-    auto const& maxGenHours = area.hydro.maxHoursGen[0];
+    // ... Getting hydro max energy
+    auto const& maxDailyGenEnergy = area.hydro.maxDailyGenEnergy[0];
 
-    hydroGenerationPermission = CheckForPositiveEnergy(maxPower, maxGenHours);
+    hydroGenerationPermission = CheckForPositiveEnergy(maxHourlyGenPower, maxDailyGenEnergy);
 
     // ---------------------
     // Hydro has inflows
@@ -120,47 +129,64 @@ AreaScratchpad::AreaScratchpad(const StudyRuntimeInfos& rinfos, Area& area) : ts
     // --------------------------
     hydroHasMod = hydroHasInflows || hydroGenerationPermission;
 
-
     // ===============
     // Pumping
     // ===============
-    // ... Hydro max power
 
-    // ... Hydro max pumping power and energy
-     auto const& maxPumpingP = area.hydro.series->maxpump;
-     auto const& maxPumpHours = area.hydro.maxHoursPump[0];
+    //  Hydro max pumping energy
+    auto const& maxDailyPumpEnergy = area.hydro.maxDailyPumpEnergy[0];
 
-    // If pumping energy is nil over the whole year, pumpHasMod is false, true otherwise.
-    pumpHasMod = CheckForPositiveEnergy(maxPumpingP, maxPumpHours);
+    //  If pumping energy is nil over the whole year, pumpHasMod is false, true otherwise.
+    pumpHasMod = CheckForPositiveEnergy(maxHourlyPumpPower, maxDailyPumpEnergy);
 }
 
 AreaScratchpad::~AreaScratchpad() = default;
 
-bool AreaScratchpad::CheckForPositiveEnergy(const Matrix<double, int32_t>& matrix,
-                                            const Matrix<double>::ColumnType& hours)
+void AreaScratchpad::CalculateMeanDailyMaxPowerMatrices(const Matrix<double>& hourlyMaxGenMatrix,
+                                                        const Matrix<double>& hourlyMaxPumpMatrix,
+                                                        uint nbOfMaxPowerTimeSeries)
 {
-    double value = 0.;
+    for (uint nbOfTimeSeries = 0; nbOfTimeSeries < nbOfMaxPowerTimeSeries; ++nbOfTimeSeries)
+    {
+        auto& hourlyMaxGenColumn = hourlyMaxGenMatrix[nbOfTimeSeries];
+        auto& hourlyMaxPumpColumn = hourlyMaxPumpMatrix[nbOfTimeSeries];
+        auto& MeanMaxDailyGenPowerColumn = meanMaxDailyGenPower[nbOfTimeSeries];
+        auto& MeanMaxDailyPumpPowerColumn = meanMaxDailyPumpPower[nbOfTimeSeries];
+
+        CalculateDailyMeanPower(hourlyMaxGenColumn, MeanMaxDailyGenPowerColumn);
+        CalculateDailyMeanPower(hourlyMaxPumpColumn, MeanMaxDailyPumpPowerColumn);
+    }
+}
 
-    for (uint width = 0; width < matrix.width; ++width)
+bool CheckForPositiveEnergy(const Matrix<double, int32_t>& power,
+                            const Matrix<double>::ColumnType& energy)
+{
+    for (uint tsNumber = 0; tsNumber < power.width; ++tsNumber)
     {
-        for (uint d = 0; d < DAYS_PER_YEAR; ++d)
+        double yearlyMaxGenEnergy = 0;
+
+        for (uint day = 0; day < DAYS_PER_YEAR; ++day)
         {
-            value += CalculateDailyMeanPower(d, matrix[width]) * hours[d];
+            yearlyMaxGenEnergy += power[tsNumber][day] * energy[day];
 
-            if (value > 0.)
+            if (yearlyMaxGenEnergy > 0.)
                 return true;
         }
-
-        value = 0.;
     }
 
     return false;
 }
 
-double CalculateDailyMeanPower(uint dYear, const Matrix<double>::ColumnType& maxPower)
+void CalculateDailyMeanPower(const Matrix<double>::ColumnType& hourlyColumn,
+                             Matrix<double>::ColumnType& dailyColumn)
 {
-    return std::accumulate(maxPower + dYear * 24, maxPower + dYear * 24 + 24, 0) / 24.;
+    for (uint day = 0; day < DAYS_PER_YEAR; ++day)
+    {
+        dailyColumn[day] = std::accumulate(hourlyColumn + day * HOURS_PER_DAY,
+                                           hourlyColumn + day * HOURS_PER_DAY + HOURS_PER_DAY,
+                                           0)
+                           / 24.;
+    }
 }
 
-} // namespace Data
-} // namespace Antares
+} // namespace Antares::Data

src/libs/antares/study/area/scratchpad.h

@@ -36,9 +36,7 @@
 #include <set>
 #include <numeric>
 
-namespace Antares
-{
-namespace Data
+namespace Antares::Data
 {
 /*!
 ** \brief Scratchpad for temporary data performed by the solver
@@ -99,14 +97,36 @@ class AreaScratchpad final
     */
     double dispatchableGenerationMargin[168];
 
+    /*!
+    ** \brief Daily mean maximum power matrices
+    **
+    ** These matrices will be calculated based on maximum
+    ** hourly generation/pumping matrices
+    */
+    Matrix<double, int32_t> meanMaxDailyGenPower;
+    Matrix<double, int32_t> meanMaxDailyPumpPower;
+
 private:
-    bool CheckForPositiveEnergy(const Matrix<double, int32_t>& matrix,
-                                const Matrix<double>::ColumnType& hours);
+    /*!
+    ** \brief Caluclation of daily mean maximum power matrices
+    **
+    ** Calculates daily mean maximum generation/pumping power
+    ** power matrices meanMaxDailyGenPower/meanMaxDailyPumpPower
+    */
+    void CalculateMeanDailyMaxPowerMatrices(const Matrix<double>& hourlyMaxGenMatrix,
+                                            const Matrix<double>& hourlyMaxPumpMatrix,
+                                            uint nbOfMaxPowerTimeSeries);
+
 }; // class AreaScratchpad
 
-double CalculateDailyMeanPower(uint dYear, const Matrix<double>::ColumnType&  maxPower);
+//  Calculates daily mean maximum generation/pumping power for one column/time-serie
+void CalculateDailyMeanPower(const Matrix<double>::ColumnType& hourlyColumn,
+                                    Matrix<double>::ColumnType& dailyColumn);
+
+//  Return true if maximum generated energy just in one day and for every TS is grated than 0
+bool CheckForPositiveEnergy(const Matrix<double, int32_t>& power,
+                            const Matrix<double>::ColumnType& energy);
 
-} // namespace Data
-} // namespace Antares
+} // namespace Antares::Data
 
 #endif // __ANTARES_LIBS_STUDY_AREA_SCRATCHPAD_H__