Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using filesystem path 2 [ANT-1999] #2454

Merged
merged 77 commits into from
Oct 16, 2024
Merged
Show file tree
Hide file tree
Changes from 73 commits
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
9c588d8
thermal areas.ini
payetvin Oct 7, 2024
c03df07
thermal area list
payetvin Oct 7, 2024
461ea8f
renew list
payetvin Oct 7, 2024
42686ea
read adq patch
payetvin Oct 8, 2024
d662e78
hydro credit
payetvin Oct 8, 2024
5336912
hydro credit
payetvin Oct 8, 2024
330a406
Merge branch 'feature/use-std-fs' of https://github.com/AntaresSimula…
payetvin Oct 8, 2024
dc39ada
hydro LoadDailyMaxEnergy
payetvin Oct 8, 2024
33a2bbe
remove using namespaces
payetvin Oct 8, 2024
c64e23b
generaldata
payetvin Oct 8, 2024
d9b65a1
format
payetvin Oct 8, 2024
7d3ef46
layers
payetvin Oct 8, 2024
30aa8d6
load crorrelation
payetvin Oct 8, 2024
617e25d
constraint
payetvin Oct 8, 2024
2153b73
fix hydro capacity
payetvin Oct 8, 2024
5cd22da
fix adq
payetvin Oct 8, 2024
f38340d
binding constraints
payetvin Oct 8, 2024
cbed456
fix constraints
payetvin Oct 8, 2024
7f6a639
xcast data
payetvin Oct 8, 2024
9b5468e
remove using yni
payetvin Oct 8, 2024
a1a231e
links
payetvin Oct 8, 2024
b8004d1
Merge remote-tracking branch 'origin/develop' into feature/use-std-fs
payetvin Oct 8, 2024
c28c0f1
in solevr 1
payetvin Oct 9, 2024
bf387aa
survey results
payetvin Oct 9, 2024
b410436
fix windows compil
payetvin Oct 9, 2024
e81cc7e
fix windows compil 2
payetvin Oct 9, 2024
3a3dc42
fix windows compil 3
payetvin Oct 9, 2024
f215166
xcast
payetvin Oct 9, 2024
b1bcbbc
test sts
payetvin Oct 9, 2024
3417282
reportbuilder 1
payetvin Oct 9, 2024
ed72ed4
reportbuilder 2
payetvin Oct 9, 2024
ee6bc3d
reportbuilder 3
payetvin Oct 9, 2024
5db13ce
reportbuilder 4
payetvin Oct 9, 2024
2db9827
fix sts windows
payetvin Oct 9, 2024
8f4471d
fix sts windows 2
payetvin Oct 9, 2024
9074df1
fix path output var
payetvin Oct 10, 2024
537d396
remove namespace using in header
payetvin Oct 10, 2024
1aa8a07
include yuni in xcast
payetvin Oct 10, 2024
a0f56fd
remove yuni include
payetvin Oct 10, 2024
7428ae5
revert 34172823169a15693abc14950ecca61f9136f841
payetvin Oct 10, 2024
a4acddf
fix compile
payetvin Oct 10, 2024
89d2d5a
xcast
payetvin Oct 10, 2024
e1aa1fc
more cleaning
payetvin Oct 10, 2024
d63c473
report builder
payetvin Oct 10, 2024
a169432
ts generator hydro
payetvin Oct 10, 2024
f887bee
solver utils
payetvin Oct 10, 2024
18cfe02
cleaning, hydro series
payetvin Oct 11, 2024
c76e5cb
more hydro
payetvin Oct 11, 2024
d071366
more cleaning
payetvin Oct 11, 2024
f27f800
fix windows
payetvin Oct 14, 2024
097168d
Revert "fix windows"
payetvin Oct 14, 2024
40b8a2d
Revert "more cleaning"
payetvin Oct 14, 2024
3fa6bd9
clean
payetvin Oct 14, 2024
ab80b1f
add todo remove with GUI to study output
payetvin Oct 14, 2024
52bc327
xcast
payetvin Oct 14, 2024
44791d4
prepro hydro
payetvin Oct 14, 2024
9166476
remove conversion from area list
payetvin Oct 14, 2024
408ab9d
store ts numbers
payetvin Oct 14, 2024
4af3689
links ts numbers
payetvin Oct 14, 2024
2ec9523
mark gui function as deprecated
payetvin Oct 15, 2024
4b96d27
format
payetvin Oct 15, 2024
7cd1eb0
add entry from buffer use path
payetvin Oct 15, 2024
9c87b88
Revert "format"
payetvin Oct 15, 2024
40dd4b0
Revert "mark gui function as deprecated"
payetvin Oct 15, 2024
7acbb35
format
payetvin Oct 15, 2024
d4df6c2
windows compile
payetvin Oct 15, 2024
b89512c
remove conversion + sep cluster list
payetvin Oct 15, 2024
f11b844
more rm useless conversion
payetvin Oct 15, 2024
5901006
yuni str conv
payetvin Oct 15, 2024
40abe46
more conv
payetvin Oct 15, 2024
0266a02
yuni str
payetvin Oct 15, 2024
9b1439d
last useless conversion
payetvin Oct 15, 2024
6e067fd
prepro
payetvin Oct 15, 2024
6bc87fc
add todo remove with gui
payetvin Oct 15, 2024
ae979cb
code smells
payetvin Oct 16, 2024
1431928
hydro container
payetvin Oct 16, 2024
2622842
format
payetvin Oct 16, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions src/analyzer/atsp/cache.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,6 @@

#include "atsp.h"

using namespace Yuni;

#define SEP Yuni::Core::IO::Separator

namespace Antares
{
void ATSP::cacheCreate()
Expand Down
2 changes: 0 additions & 2 deletions src/libs/antares/memory/memory.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,6 @@

using namespace Yuni;

#define SEP Yuni::IO::Separator

namespace Antares
{
/*extern*/
Expand Down
11 changes: 0 additions & 11 deletions src/libs/antares/paths/include/antares/paths/list.h
Original file line number Diff line number Diff line change
Expand Up @@ -178,17 +178,6 @@ class PathList
return item.empty();
}

template<class StringT>
size_t sizeOnDisk(const StringT& sourceFolder) const
{
if (item.empty())
{
return 0;
}
pTmp = sourceFolder;
return (!pTmp) ? 0 : internalSizeOnDisk();
}

Comment on lines -181 to -191
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to other readers: this function is unused

size_t totalSizeInBytes() const;

template<class StringT>
Expand Down
18 changes: 1 addition & 17 deletions src/libs/antares/paths/list.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -41,23 +41,6 @@ void PathList::clear()
item.clear();
}

size_t PathList::internalSizeOnDisk() const
{
size_t size = 0;
Clob buffer;

const ItemList::const_iterator end = item.end();
for (ItemList::const_iterator i = item.begin(); i != end; ++i)
{
if (!(i->second.options & pathListOptFolder))
{
buffer.clear() << pTmp << SEP << i->first;
size += (size_t)IO::File::Size(pTmp);
}
}
return size;
}

Comment on lines -44 to -60
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: this function is unused.

size_t PathList::totalSizeInBytes() const
{
size_t size = 0;
Expand All @@ -70,6 +53,7 @@ size_t PathList::totalSizeInBytes() const
return size;
}

// TODO VP: remove with tools
payetvin marked this conversation as resolved.
Show resolved Hide resolved
uint PathList::internalDeleteAllEmptyFolders()
{
if (!pTmp || item.empty())
Expand Down
8 changes: 2 additions & 6 deletions src/libs/antares/series/series.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,6 @@

#include <antares/antares/constants.h>

using namespace Yuni;

#define SEP IO::Separator

namespace Antares::Data
{
void TimeSeriesNumbers::registerSeries(const TimeSeries* s, std::string label)
Expand Down Expand Up @@ -135,8 +131,8 @@ int TimeSeries::saveToFolder(const std::string& areaID,
const std::string& folder,
const std::string& prefix) const
{
Clob buffer;
buffer.clear() << folder << SEP << prefix << areaID << ".txt";
Yuni::Clob buffer;
buffer.clear() << folder << Yuni::IO::Separator << prefix << areaID << ".txt";
return timeSeries.saveToCSVFile(buffer, 0);
}

Expand Down
50 changes: 22 additions & 28 deletions src/libs/antares/study/area/links.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -38,11 +38,6 @@ namespace fs = std::filesystem;

#define SEP (IO::Separator)

namespace
{
const YString DIRECTORY_NAME_FOR_TRANSMISSION_CAPACITIES = "ntc";
}

namespace Antares
{
namespace Data
Expand Down Expand Up @@ -78,15 +73,14 @@ AreaLink::~AreaLink()
{
}

bool AreaLink::linkLoadTimeSeries_for_version_below_810(const AnyString& folder)
bool AreaLink::linkLoadTimeSeries_for_version_below_810(const fs::path& folder)
{
String buffer;
buffer.clear() << folder << SEP << with->id << ".txt";
fs::path path = folder / std::string(with->id + ".txt");

// Load link's data
Matrix<> tmpMatrix;
const uint matrixWidth = 8;
if (!tmpMatrix.loadFromCSVFile(buffer,
if (!tmpMatrix.loadFromCSVFile(path.string(),
matrixWidth,
HOURS_PER_YEAR,
Matrix<>::optFixedSize | Matrix<>::optImmediate))
Expand All @@ -110,26 +104,28 @@ bool AreaLink::linkLoadTimeSeries_for_version_below_810(const AnyString& folder)
return true;
}

bool AreaLink::linkLoadTimeSeries_for_version_820_and_later(const AnyString& folder)
bool AreaLink::linkLoadTimeSeries_for_version_820_and_later(const fs::path& folder)
{
String capacitiesFolder;
capacitiesFolder << folder << SEP << "capacities";

String filename;
bool success = true;

// Read link's parameters times series
filename.clear() << folder << SEP << with->id << "_parameters.txt";
success = parameters.loadFromCSVFile(filename, fhlMax, HOURS_PER_YEAR, Matrix<>::optFixedSize)
std::string paramId = with->id + "_parameters.txt";
fs::path path = folder / paramId;
success = parameters.loadFromCSVFile(path.string(),
fhlMax,
HOURS_PER_YEAR,
Matrix<>::optFixedSize)
&& success;

fs::path capacitiesFolder = folder / "capacities";

// Read link's direct capacities time series
filename.clear() << capacitiesFolder << SEP << with->id << "_direct.txt";
success = directCapacities.loadFromFile(filename.c_str(), false) && success;
path = capacitiesFolder / std::string(with->id + "_direct.txt");
success = directCapacities.loadFromFile(path, false) && success;

// Read link's indirect capacities time series
filename.clear() << capacitiesFolder << SEP << with->id << "_indirect.txt";
success = indirectCapacities.loadFromFile(filename.c_str(), false) && success;
path = capacitiesFolder / std::string(with->id + "_indirect.txt");
success = indirectCapacities.loadFromFile(path, false) && success;

return success;
}
Expand Down Expand Up @@ -173,7 +169,7 @@ void AreaLink::overrideTransmissionCapacityAccordingToGlobalParameter(
}
}

bool AreaLink::loadTimeSeries(const StudyVersion& version, const AnyString& folder)
bool AreaLink::loadTimeSeries(const StudyVersion& version, const fs::path& folder)
{
if (version < StudyVersion(8, 2))
{
Expand All @@ -187,14 +183,12 @@ bool AreaLink::loadTimeSeries(const StudyVersion& version, const AnyString& fold

void AreaLink::storeTimeseriesNumbers(Solver::IResultWriter& writer) const
{
Clob path;
std::string buffer;

path << "ts-numbers" << SEP << DIRECTORY_NAME_FOR_TRANSMISSION_CAPACITIES << SEP << from->id
<< SEP << with->id << ".txt";
std::string filename = with->id + ".txt";
fs::path path = fs::path("ts-numbers") / "ntc" / from->id.to<std::string>() / filename;

std::string buffer;
timeseriesNumbers.saveToBuffer(buffer);
writer.addEntryFromBuffer(path.c_str(), buffer);
writer.addEntryFromBuffer(path, buffer);
}

void AreaLink::detach()
Expand Down Expand Up @@ -527,7 +521,7 @@ bool AreaLinksLoadFromFolder(Study& study, AreaList* areaList, Area* area, const
link.comments.clear();
link.displayComments = true;

ret = link.loadTimeSeries(study.header.version, folder.string()) && ret;
ret = link.loadTimeSeries(study.header.version, folder) && ret;

// Checks on loaded link's data
if (study.usedByTheSolver)
Expand Down
92 changes: 44 additions & 48 deletions src/libs/antares/study/area/list.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ namespace Antares::Data
{
namespace // anonymous
{
static bool AreaListLoadThermalDataFromFile(AreaList& list, const Clob& filename)
static bool AreaListLoadThermalDataFromFile(AreaList& list, const fs::path& filename)
{
// Reset to 0
list.each(
Expand Down Expand Up @@ -776,38 +776,39 @@ bool AreaList::saveToFolder(const AnyString& folder) const
return ret;
}

template<class StringT>
static void readAdqPatchMode(Study& study, Area& area, StringT& buffer)
static void readAdqPatchMode(Study& study, Area& area)
{
if (study.header.version >= StudyVersion(8, 3))
if (study.header.version < StudyVersion(8, 3))
{
return;
}

fs::path adqPath = study.folderInput / "areas" / area.id.to<std::string>()
/ "adequacy_patch.ini";
IniFile ini;
if (ini.open(adqPath))
{
buffer.clear() << study.folderInput << SEP << "areas" << SEP << area.id << SEP
<< "adequacy_patch.ini";
IniFile ini;
if (ini.open(buffer))
auto* section = ini.find("adequacy-patch");
for (auto* p = section->firstProperty; p; p = p->next)
{
auto* section = ini.find("adequacy-patch");
for (auto* p = section->firstProperty; p; p = p->next)
CString<30, false> tmp;
tmp = p->key;
tmp.toLower();
if (tmp == "adequacy-patch-mode")
{
CString<30, false> tmp;
tmp = p->key;
tmp.toLower();
if (tmp == "adequacy-patch-mode")
{
auto value = (p->value).toLower();
auto value = (p->value).toLower();

if (value == "virtual")
{
area.adequacyPatchMode = Data::AdequacyPatch::virtualArea;
}
else if (value == "inside")
{
area.adequacyPatchMode = Data::AdequacyPatch::physicalAreaInsideAdqPatch;
}
else
{
area.adequacyPatchMode = Data::AdequacyPatch::physicalAreaOutsideAdqPatch;
}
if (value == "virtual")
{
area.adequacyPatchMode = Data::AdequacyPatch::virtualArea;
}
else if (value == "inside")
{
area.adequacyPatchMode = Data::AdequacyPatch::physicalAreaInsideAdqPatch;
}
else
{
area.adequacyPatchMode = Data::AdequacyPatch::physicalAreaOutsideAdqPatch;
}
}
}
Expand Down Expand Up @@ -899,7 +900,7 @@ static bool AreaListLoadFromFolderSingleArea(Study& study,
{
// if changes are required, please update reloadXCastData()
fs::path loadPath = study.folderInput / "load" / "prepro" / area.id.to<std::string>();
ret = area.load.prepro->loadFromFolder(loadPath.string()) && ret;
ret = area.load.prepro->loadFromFolder(loadPath) && ret;
}
if (!options.loadOnlyNeeded || !area.load.prepro) // Series
{
Expand All @@ -916,7 +917,7 @@ static bool AreaListLoadFromFolderSingleArea(Study& study,
{
// if changes are required, please update reloadXCastData()
fs::path solarPath = study.folderInput / "solar" / "prepro" / area.id.to<std::string>();
ret = area.solar.prepro->loadFromFolder(solarPath.string()) && ret;
ret = area.solar.prepro->loadFromFolder(solarPath) && ret;
}
if (!options.loadOnlyNeeded || !area.solar.prepro) // Series
{
Expand All @@ -940,7 +941,7 @@ static bool AreaListLoadFromFolderSingleArea(Study& study,
{
// if changes are required, please update reloadXCastData()
fs::path hydroPrepro = pathHydro / "prepro";
ret = area.hydro.prepro->loadFromFolder(study, area.id, hydroPrepro.string()) && ret;
ret = area.hydro.prepro->loadFromFolder(study, area.id, hydroPrepro) && ret;
ret = area.hydro.prepro->validate(area.id) && ret;
}

Expand Down Expand Up @@ -972,7 +973,7 @@ static bool AreaListLoadFromFolderSingleArea(Study& study,
{
// if changes are required, please update reloadXCastData()
fs::path windPath = study.folderInput / "wind" / "prepro" / area.id.to<std::string>();
ret = area.wind.prepro->loadFromFolder(windPath.string()) && ret;
ret = area.wind.prepro->loadFromFolder(windPath) && ret;
}
if (!options.loadOnlyNeeded || !area.wind.prepro) // Series
{
Expand Down Expand Up @@ -1016,7 +1017,7 @@ static bool AreaListLoadFromFolderSingleArea(Study& study,
}

// Adequacy patch
readAdqPatchMode(study, area, buffer);
readAdqPatchMode(study, area);

// Nodal Optimization
fs::path nodalPath = study.folderInput / "areas" / area.id.to<std::string>()
Expand Down Expand Up @@ -1160,21 +1161,18 @@ bool AreaList::loadFromFolder(const StudyLoadOptions& options)
// Thermal data, specific to areas
{
logs.info() << "Loading thermal clusters...";
buffer.clear() << pStudy.folderInput << SEP << "thermal" << SEP << "areas.ini";
ret = AreaListLoadThermalDataFromFile(*this, buffer) && ret;
fs::path thermalPath = pStudy.folderInput / "thermal";
fs::path areaIniPath = thermalPath / "areas.ini";
ret = AreaListLoadThermalDataFromFile(*this, areaIniPath) && ret;

// The cluster list must be loaded before the method
// ensureDataIsInitialized is called
// The cluster list must be loaded before the method ensureDataIsInitialized is called
// in order to allocate data with all thermal clusters.
CString<30, false> thermalPlant;
thermalPlant << SEP << "thermal" << SEP << "clusters" << SEP;

auto end = areas.end();
for (auto i = areas.begin(); i != end; ++i)
{
Area& area = *(i->second);
buffer.clear() << pStudy.folderInput << thermalPlant << area.id;
ret = area.thermal.list.loadFromFolder(pStudy, buffer.c_str(), &area) && ret;
fs::path areaPath = thermalPath / "clusters" / area.id.to<std::string>();
ret = area.thermal.list.loadFromFolder(pStudy, areaPath, &area) && ret;
ret = area.thermal.list.validateClusters(pStudy.parameters) && ret;
}
}
Expand Down Expand Up @@ -1203,18 +1201,16 @@ bool AreaList::loadFromFolder(const StudyLoadOptions& options)
// Renewable data, specific to areas
if (studyVersion >= StudyVersion(8, 1))
{
// The cluster list must be loaded before the method
// ensureDataIsInitialized is called
// The cluster list must be loaded before the method ensureDataIsInitialized is called
// in order to allocate data with all renewable clusters.
CString<30, false> renewablePlant;
renewablePlant << SEP << "renewables" << SEP << "clusters" << SEP;
fs::path renewClusterPath = pStudy.folderInput / "renewables" / "clusters";

auto end = areas.end();
for (auto i = areas.begin(); i != end; ++i)
{
Area& area = *(i->second);
buffer.clear() << pStudy.folderInput << renewablePlant << area.id;
ret = area.renewable.list.loadFromFolder(buffer.c_str(), &area) && ret;
fs::path areaPath = renewClusterPath / area.id.to<std::string>();
ret = area.renewable.list.loadFromFolder(areaPath, &area) && ret;
ret = area.renewable.list.validateClusters() && ret;
}
}
Expand Down
Loading
Loading