This module is based on powsybl-core network modeling and framework.
It provides two main tools : MappingTool and MetrixTool
It can also be used as a library to integrate its functionalities
into java applications.
The MappingTool provide a mapping configuration from csv timeseries to network. This configuration allows to generate iidm network at specified time. Furthermore, it allows using the second tool, MetrixTool.
MetrixTool can compute power load flow and optimal power load flow for the multi variant networks generated by the mapping tool. Its output are completed by synthesis.
MetrixTool calls metrix simulator as an external program. This program has its own requirements.
The following status are produced by the reduced CI coverage tests
Run ./install.sh
itools mapping and itools metrix are the command line tools that are provided to use and test the mapping and
metrix.
Their respective usage is detailed in the following sections.
For more advanced and intensive usage it is recommended to embed this package as a library as the time series storage is
not optimized.
Powsybl AFS can be used for that matter.
To use the mapping, add com.powsybl:powsybl-metrix-mapping module to your dependencies.
Then you need :
- a case file
- a mapping groovy script configuration
- a time series store
// Entry timeseries
InMemoryTimeSeriesStore store = new InMemoryTimeSeriesStore();
store.importTimeSeries(Collections.singletonList(Paths.get("/path/to/timeseries.csv")));
// Mapping file
Path mappingFile = Paths.get("path/to/mappingFile");
// Network
Network network = NetworkXml.read(Paths.get("/path/to/network.xiidm"));
// Computation parameters
MappingParameters mappingParameters = MappingParameters.load();
int firstVariant = ...
int maxVariantCount = ...
ComputationRange computationRange = new ComputationRange(store.getTimeSeriesDataVersions(), firstVariant, maxVariantCount);
// Generate mapping configuration
TimeSeriesMappingConfig config;
try (Reader reader = Files.newBufferedReader(mappingFile, StandardCharsets.UTF_8)) {
TimeSeriesDslLoader dslLoader = new TimeSeriesDslLoader(reader, mappingFile.getFileName().toString());
config = dslLoader.load(network, mappingParameters, store, new DataTableStore(), computationRange);
}
// Export result in csv
TimeSeriesMappingConfigCsvWriter csvWriter = new TimeSeriesMappingConfigCsvWriter(config, network, store, computationRange, mappingParameters.getWithTimeSeriesStats());
csvWriter.writeMappingCsv(Paths.get("/path/to/output"));
// Compute mapping on network
TimeSeriesMappingLogger logger = new TimeSeriesMappingLogger();
List<TimeSeriesMapperObserver> observers = new ArrayList<>();
// Add network generation computation
DataSource dataSource = DataSourceUtil.createDataSource(Paths.get("/path/to/networkOutputDir"), network.getId(), null);
observers.add(new NetworkPointWriter(network, dataSource));
// Add timeseries mapping export
TimeSeriesIndex index = new TimeSeriesMappingConfigTableLoader(config,store).checkIndexUnicity();
int lastPoint = Math.min(firstVariant + maxVariantCount, index.getPointCount()) - 1;
Range<Integer> range = Range.closed(firstVariant, lastPoint);
observers.add(new EquipmentTimeSeriesWriterObserver(network, config, maxVariantCount, range, Paths.get("/path/to/equipmentTimeSeriesDir")));
observers.add(new EquipmentGroupTimeSeriesWriterObserver(network, config, maxVariantCount, range, Paths.get("/path/to/equipmentTimeSeriesDir")));
// Apply mapping to network
TimeSeriesMapperParameters parameters = new TimeSeriesMapperParameters(store.getTimeSeriesDataVersions(), range, true, true, false, mappingParameters.getToleranceThreshold());
TimeSeriesMapper mapper = new TimeSeriesMapper(config, parameters, network, logger);
mapper.mapToNetwork(store, observers);Further documentation is available on the dedicated page on our website.
To use metrix with java, add com.powsybl:powsybl-metrix-integration module to your dependencies.
Then you need :
- a case file
- a mapping groovy script configuration
- a time series store
- a metrix configuration script
- (optional) a contingency configuration script
- (optional) a remedial actions configuration file
ComputationManager computationManager = LocalComputationManager.getDefault();
// Network
NetworkSource networkSource = new DefaultNetworkSourceImpl(Paths.get("/path/to/case.xiidm"), computationManager)
// Timeseries
InMemoryTimeSeriesStore store = new InMemoryTimeSeriesStore();
store.importTimeSeries(Collections.singletonList(Paths.get("/path/to/timeseries.csv")));
// Contingencies
ContingenciesProvider contingenciesProvider = new GroovyDslContingenciesProvider(Paths.get("/path/to/contingencies.groovy"));
// Mapping
Supplier<Reader> mappingReader = () -> Files.newBufferedReader(Paths.get("/path/to/mapping.groovy"), StandardCharsets.UTF_8);
// Metrix config
Supplier<Reader> metrixDslReader = () -> Files.newBufferedReader(Paths.get("/path/to/metrixConfig.groovy"), StandardCharsets.UTF_8);
// Remedial actions
Supplier<Reader> remedialActionsReader = () -> Files.newBufferedReader(Paths.get("/path/to/remedialActions.txt"), StandardCharsets.UTF_8);
// Result timeseries store
FileSystemTimeSeriesStore resultStore = new FileSystemTimeSeriesStore(Paths.get("/path/to/outputdir"));
// Result listener
ResultListener listener = new ResultListener() {
@Override
public void onChunkResult(int version, int chunk, List<TimeSeries> timeSeriesList, Network networkPoint){
resultStore.importTimeSeries(timeSeriesList, version, false);
}
}
// Run metrix configuration analysis
MetrixAnalysis metrixAnalysis = new MetrixAnalysis(networkSource, mappingReader, metrixDslReader, remedialActionsReader, contingenciesProvider, store, logger, computationRange);
MetrixAnalysisResult analysisResult = metrixAnalysis.runAnalysis("extern tool");
// Run metrix
Metrix metrix = new Metrix(remedialActionsReader, store, resultStore, logArchive, computationManager, logger, analysisResult)
MetrixRunParameters runParams = new MetrixRunParameters(firstVariant, variantCount, versions, chunkSize, true, true, false);
metrix.run(runParams, listener);Further documentation is available on the dedicated page on our website.
Metrix simulator is an independent C++ executable. It must be installed before using powsybl-metrix.
It has its own toolchain and requirements.
inputs:
- fort.json: aggregate file containing network data, mapping and metrix options. The name of this file is frozen
- variant file: this file describes the network modification to apply to the information present in the fort.json file for each defined variant. The name of this name is chosen at metrix launch (see --help)
- counter file: this file describes the list of topological actions that are allowed as preemptive actions to counter an issue. The name of this name can be chosen at metrix launch (see --help)
- index first variant: the first variant to process in the variant file. Is chosen at launch.
- number of variants: the number of variants to process. Is chosen at launch.
outputs:
- user log file: the user log file, displaying user-friendly information about the finished run. This information is duplicated in the developer log file
- developer log files: a set of more development-axed log files, with more information
- result files: the result files (1 by variant processed)
all input options are described in --help option
To build metrix-simulator, you need:
- A C++ compiler that supports C++11 (clang 3.3 or higher, g++ 5.0 or higher)
- CMake (3.14 or higher)
- Make
- Boost development packages (1.66 or higher)
$> apt install -y cmake g++ git libboost-all-dev libxml2-dev make
$> apt install -y g++ git libboost-all-dev libxml2-dev make wget
Note: Under Ubuntu 18.04, the default CMake package is too old (3.10), so you have to install it manually:
$> wget https://cmake.org/files/v3.14/cmake-3.14.0-Linux-x86_64.tar.gz
$> tar xzf cmake-3.14.0-Linux-x86_64.tar.gz
$> export PATH=$PWD/cmake-3.14.0-Linux-x86_64/bin:$PATH
$> yum install -y boost-devel gcc-c++ git libxml2-devel make wget
Note: Under CentOS 8, the default CMake package is too old (3.11.4), so you have to install it manually:
$> wget https://cmake.org/files/v3.14/cmake-3.14.0-Linux-x86_64.tar.gz
$> tar xzf cmake-3.14.0-Linux-x86_64.tar.gz
$> export PATH=$PWD/cmake-3.14.0-Linux-x86_64/bin:$PATH
$> yum install -y gcc-c++ git libxml2-devel make wget
Note: Under CentOS 7, the default boost-devel package is too old (1.53), so we install Boost 1.66 from epel-release.
$> yum install -y epel-release
$> yum install -y boost166-devel
$> export BOOST_INCLUDEDIR=/usr/include/boost166
$> export BOOST_LIBRARYDIR=/usr/lib64/boost166
Note: Under CentOS 7, the default CMake package is too old (2.8.12), so you have to install it manually:
$> wget https://cmake.org/files/v3.14/cmake-3.14.0-Linux-x86_64.tar.gz
$> tar xzf cmake-3.14.0-Linux-x86_64.tar.gz
$> export PATH=$PWD/cmake-3.14.0-Linux-x86_64/bin:$PATH
1 - Clone the project
$> git clone https://github.com/powsybl/powsybl-metrix.git
$> cd powsybl-metrix/metrix-simulator
2 - Build the project, with 3rd parties First build the 3rd parties
$> mkdir build
$> mkdir build/external
$> cd build/external
$> cmake ../../external -DCMAKE_BUILD_TYPE=<BUILD_TYPE_3PARTIES>
$> cmake --build .
Then build the executable
$> cd ..
$> cmake .. -DCMAKE_INSTALL_PREFIX=<PREFIX> -DCMAKE_BUILD_TYPE=<BUILD_TYPE>
$> cmake --build . --target install
The following CMAKE options can be set for the executable configuration:
- USE_SIRIUS_SHARED (default = OFF): If active, project will link using the shared library of sirius solver instead of static library
- METRIX_RUN_ALL_TESTS (default = ON): If inactive, projects will execute a reduced scope of tests
This project uses clang-tidy to verify the code style. This tool is provided with the clang extra tools. To enable the code style checking, add the -DCMAKE_CXX_CLANG_TIDY=clang-tidy flag to the configure command.
A clang-format file is also provided to format the code by using clang-format. Most IDEs have an option to format files using clang-format automatically.
This project uses either gcov or llvm-cov to compute the code coverage. We also use gcovr (4.2 or higher) to generate both sonar and HTML reports. To compute the code coverage, add the -DCODE_COVERAGE=TRUE flag to the configure command.
The following environments variables must be defined in order to run metrix properly:
METRIX_ETC: location of the .dic files for language region (some of these dictionaries are exported in etc directory in install directory)
All options are detailed in the helper
$> ./metrix-simulator --help
Usage:
metrix-simulator <errorFilepath> <variantFilepath> <resultsFilepath> <firstVariantIndex> <numberVariants> <paradesFilepath>
<paradesFilepath> = "parades.csv" by default
[options]
Metrix options:
-h [ --help ] Display help message
--log-level arg Logger level (allowed values are critical,
error, warning, info, debug, trace): default is
info
-p [ --print-log ] Print developer log in standard output
--verbose-config Activate debug/trace logs relative to
configuration
--verbose-constraints Activate debug/trace logs relative to
constraint detection
--write-constraints Write the constraints in a dedicated file
--print-constraints Trace in logs the constraints matrix (time
consuming even if trace logs are not active),
log level at trace is required
--write-sensitivity Write the sensivity matrix in a dedicated file
--write-report Write the rate matrix report in a dedicated
file
--check-constraints-level arg Check adding constraints:
0: no check (default)
1: When adding a constraint, perform a load
flow to check transit (more time consuming)
2: When adding a constraint, run every incident
to check that we didn't forget a constraint
(even more time consuming
--compare-reports Compare load flow reports after application of
report factors to check trigger of coupling
--no-incident-group Ignore incident if a group of N-K is not
available
--all-outputs Display all values in results files
--mps-file Export MPS file