CGMES: remove extension for Control Areas, use IIDM Area #3149

zamarrenolm · 2024-09-20T06:36:15Z

Please check if the PR fulfills these requirements

The commit message follows our guidelines
Tests for the changes have been added (for bug fixes / features)
Docs have been added / updated (for bug fixes / features)

Does this PR already have an issue describing the problem?

What kind of change does this PR introduce?

Update

What is the current behavior?

Information from CGMES control areas is stored using an extension to the Network Model (CgmesControlArea).

What is the new behavior (if this is a feature change)?
Information from CGMES control areas is stored using native IIDM objects of class Area.
CGMES pTolerance attribute is stored as a property.
The ENTSO-E attribute IdentifiedObject.energyIdentCodeEic is stored as an alias.

Does this PR introduce a breaking change or deprecate an API?

Yes
No

If yes, please check if the following requirements are fulfilled

The Breaking Change or Deprecated label has been added
The migration steps are described in the following section

What changes might users need to make in their application due to this PR? (migration steps)

Other information:

Check if the balancing adjustment mechanism has to be updated (in powsybl-entsoe) and adapt it if needed
Adapt UCTE import/export
Ensure backward compatibility (being able to read the old extension)

Signed-off-by: Luma <[email protected]>

rcourtier · 2024-11-06T09:01:37Z

cgmes/cgmes-conversion/src/main/java/com/powsybl/cgmes/conversion/Conversion.java

-    private static void createControlArea(CgmesControlAreas cgmesControlAreas, PropertyBag ca) {
-        String controlAreaId = ca.getId("ControlArea");
-        cgmesControlAreas.newCgmesControlArea()
+    private static void createControlArea(Context context, PropertyBag ca) {


I suggest you create an AreaConversion.java class in com.powsybl.cgmes.conversion.elements package and move that piece of code there.

Code moved. Had to make public a class in conversion, related to the mapping of terminals of tie flows

cgmes/cgmes-conversion/src/main/java/com/powsybl/cgmes/conversion/Conversion.java

rcourtier · 2024-11-06T09:09:13Z

cgmes/cgmes-conversion/src/main/java/com/powsybl/cgmes/conversion/Conversion.java

            context.pushReportNode(CgmesReports.convertingElementTypeReport(reportNode, CgmesNames.CONTROL_AREA));
-            network.newExtension(CgmesControlAreasAdder.class).add();
-            CgmesControlAreas cgmesControlAreas = network.getExtension(CgmesControlAreas.class);
-            cgmes.controlAreas().forEach(ca -> createControlArea(cgmesControlAreas, ca));
-            cgmes.tieFlows().forEach(tf -> addTieFlow(context, cgmesControlAreas, tf));
-            cgmesControlAreas.cleanIfEmpty();
+            cgmes.controlAreas().forEach(ca -> createControlArea(context, ca));
+            cgmes.tieFlows().forEach(tf -> addTieFlow(context, tf));
            context.popReportNode();


You could create an AreaConversion and a TieFlowConversion class (see comments below) and rely on the convert method here (just like other network elements are converted).

specific classes created

rcourtier · 2024-11-06T10:09:16Z

...s/cgmes-conversion/src/main/java/com/powsybl/cgmes/conversion/export/CgmesExportContext.java

+        // If no control area exists, create one for the whole network, containing the dangling lines as boundaries,
+        // but only if the network does not contain subnetworks
+        long numControlAreas = network.getAreaStream().filter(a -> a.getAreaType().equals("ControlAreaTypeKind.Interchange")).count();
+        long numSubnetworks = network.getSubnetworks().size();


An int could suffice since size() returns an int anyway.

rcourtier · 2024-11-06T10:41:41Z

cgmes/cgmes-conversion/src/main/java/com/powsybl/cgmes/conversion/Conversion.java

            context.ignored("Tie Flow", String.format("Tie Flow %s refers to a non-existing control area", tf.getId("TieFlow")));
            return;
        }
        String terminalId = tf.getId("terminal");
        Boundary boundary = context.terminalMapping().findBoundary(terminalId, context.cgmes());
        if (boundary != null) {
-            cgmesControlArea.add(boundary);
+            area.newAreaBoundary()
+                    .setAc(true)


Isn't it possible to set the correct value for this boolean here in CGMES import, the same way it is done in the CGMES export?

Done. Added a specific unit test.

rcourtier · 2024-11-07T09:22:26Z

...s-conversion/src/test/java/com/powsybl/cgmes/conversion/test/export/EquipmentExportTest.java

-        exportOptions.setExtensions(Collections.emptySet());
+        exportOptions.setExtensions(Set.of("cgmesControlAreas"));


rcourtier · 2024-11-07T09:22:34Z

...s-conversion/src/test/java/com/powsybl/cgmes/conversion/test/export/EquipmentExportTest.java

+        for (Area area : network.getAreas()) {
+            area.setInterchangeTarget(0.0);
+        }


rcourtier · 2024-11-07T09:23:39Z

...gmes-conversion/src/test/java/com/powsybl/cgmes/conversion/test/export/ExportXmlCompare.java

-                if (control.getValue().toString().equals("netInterchange")
+                if (control.getValue() != null && control.getValue().toString().equals("netInterchange")


That part could be removed when the issue of the CGMES export altering the IIDM network will be tackled.

rcourtier · 2024-11-07T09:24:43Z

...version/src/test/java/com/powsybl/cgmes/conversion/test/export/StateVariablesExportTest.java

+        // Also, if a default area was created during export to CGMES, remove it from the expected network
+        // The actual network will not have it, it is the original network directly exported as IIDM
+        if (!originalNetworkHasAreas) {
+            expected.getAreaStream().map(Area::getId).toList().forEach(a -> expected.getArea(a).remove());
+        }


That part could be removed when the issue of the CGMES export altering the IIDM network will be tackled.

rcourtier · 2024-11-07T09:32:10Z

cgmes/cgmes-conversion/src/main/java/com/powsybl/cgmes/conversion/Conversion.java

-        RegulatingTerminalMapper.mapForTieFlow(terminalId, context).ifPresent(cgmesControlArea::add);
+        RegulatingTerminalMapper.mapForTieFlow(terminalId, context)
+                .ifPresent(t -> area.newAreaBoundary()
+                        .setAc(true)


Isn't it possible to set the correct value for this boolean here in CGMES import, the same way it is done in the CGMES export?

Signed-off-by: Luma <[email protected]>

…is used Signed-off-by: Luma <[email protected]>

sonarqubecloud · 2024-12-16T16:57:54Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
88.4% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

rcourtier · 2024-12-18T14:45:28Z

.../cgmes-conversion/src/main/java/com/powsybl/cgmes/conversion/elements/TieFlowConversion.java

+        String controlAreaId = p.getId(CgmesNames.CONTROL_AREA);
+        Area area = context.network().getArea(controlAreaId);
+        if (area == null) {
+            context.ignored("Tie Flow", String.format("Tie Flow %s refers to a non-existing control area", p.getId("TieFlow")));


You could use the constant from CgmesNames for more consistency:

p.getId(CgmesNames.TIE_FLOW)

rcourtier · 2024-12-18T14:45:37Z

.../cgmes-conversion/src/main/java/com/powsybl/cgmes/conversion/elements/TieFlowConversion.java

+            context.ignored("Tie Flow", String.format("Tie Flow %s refers to a non-existing control area", p.getId("TieFlow")));
+            return;
+        }
+        String terminalId = p.getId("terminal");


You could use the constant from CgmesNames for more consistency:

p.getId(CgmesNames.TERMINAL)

cgmes/cgmes-conversion/src/main/java/com/powsybl/cgmes/conversion/Conversion.java

rcourtier · 2024-12-18T15:56:08Z

...onversion/src/main/java/com/powsybl/cgmes/conversion/export/SteadyStateHypothesisExport.java

+        double pTolerance;
+        if (controlArea.hasProperty("pTolerance")) {
+            pTolerance = Double.parseDouble(controlArea.getProperty("pTolerance"));


Now that you have removed the else clause, you can merge the declaration of variable pTolerance with its initialization.

...onversion/src/main/java/com/powsybl/cgmes/conversion/export/SteadyStateHypothesisExport.java

rcourtier · 2024-12-18T16:00:17Z

cgmes/cgmes-conversion/src/main/java/com/powsybl/cgmes/conversion/CgmesExport.java

+
+        String controlAreaId = context.getNamingStrategy().getCgmesId(refTyped(network), CONTROL_AREA);
+        Area area = network.newArea()
+                .setAreaType("ControlAreaTypeKind.Interchange")


ControlAreaTypeKind.Interchange enum value is used here as well as in:

EquipmentExport

SteadyStateHypothesisExport

ControlAreaEq

CgmesConformity1ModifiedConversionTest

EquipmentExportTest

SteadyStateHypothesisExportTest

I think it would be worth creating a constant in CgmesNames.

zamarrenolm added 2 commits September 20, 2024 08:27

CGMES: remove extension for Control Areas, use iidm Area

9bc3e5c

Signed-off-by: Luma <[email protected]>

add energy identification code EIC as alias

f164ae5

Signed-off-by: Luma <[email protected]>

zamarrenolm added the CGMES label Sep 20, 2024

zamarrenolm added 10 commits September 20, 2024 17:08

do not delete existing cgmes extension

8831363

Signed-off-by: Luma <[email protected]>

copy the cgmes control area type to the iidm area type

38c02c9

Signed-off-by: Luma <[email protected]>

use iidm area for cgmes export of control areas, adjust unit tests

df99a96

Signed-off-by: Luma <[email protected]>

always set pTolerance; adjust unit tests

9387a35

Signed-off-by: Luma <[email protected]>

Merge branch 'main' into remove_cgmes_control_areas_extension

5019069

Signed-off-by: Luma <[email protected]>

Merge branch 'main' into remove_cgmes_control_areas_extension

c67ca0c

EIC code and target for default control area created for export

69026ac

Signed-off-by: Luma <[email protected]>

Merge branch 'main' into remove_cgmes_control_areas_extension

e946c3f

p tolerance positive, by default 1% of net interchange

20cd5e5

Signed-off-by: Luma <[email protected]>

refactor writing of tie flows in equipment export

ee15e39

Signed-off-by: Luma <[email protected]>

zamarrenolm marked this pull request as ready for review October 25, 2024 12:09

zamarrenolm requested a review from rcourtier October 25, 2024 13:41

determine if a boundary should be ac or dc based on the reference data

0af40d1

Signed-off-by: Luma <[email protected]>

rcourtier requested changes Nov 7, 2024

View reviewed changes

zamarrenolm added 2 commits November 14, 2024 14:08

move control area conversion to specific class

f51309f

Signed-off-by: Luma <[email protected]>

int for size of list

53cb10d

Signed-off-by: Luma <[email protected]>

alicecaron added the December 2024 release label Nov 14, 2024

import tie flow sets the ac/dc type

81b63e3

Signed-off-by: Luma <[email protected]>

So-Fras removed the December 2024 release label Dec 4, 2024

zamarrenolm added 3 commits December 12, 2024 17:08

Merge branch 'main' into remove_cgmes_control_areas_extension

f2a31ac

remove legacy extension, adapt unit tests

c081ba2

Signed-off-by: Luma <[email protected]>

remove local directory

cc60a53

Signed-off-by: Luma <[email protected]>

So-Fras added the Release 03/2025 label Dec 16, 2024

zamarrenolm added 2 commits December 16, 2024 16:59

explicit creation of default control area of type interchange

4f9d471

Signed-off-by: Luma <[email protected]>

control area net interchange tolerance is optional, no default value …

d264397

…is used Signed-off-by: Luma <[email protected]>

rcourtier reviewed Dec 18, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CGMES: remove extension for Control Areas, use IIDM Area #3149

CGMES: remove extension for Control Areas, use IIDM Area #3149

zamarrenolm commented Sep 20, 2024 •

edited by So-Fras

Loading

rcourtier Nov 6, 2024

zamarrenolm Nov 14, 2024

rcourtier Nov 6, 2024

zamarrenolm Nov 14, 2024

rcourtier Nov 6, 2024

zamarrenolm Nov 14, 2024

rcourtier Nov 6, 2024

zamarrenolm Nov 14, 2024

rcourtier Nov 7, 2024

rcourtier Nov 7, 2024

rcourtier Nov 7, 2024

rcourtier Nov 7, 2024

rcourtier Nov 7, 2024

sonarqubecloud bot commented Dec 16, 2024

rcourtier Dec 18, 2024

rcourtier Dec 18, 2024

rcourtier Dec 18, 2024

rcourtier Dec 18, 2024 •

edited

Loading

		exportOptions.setExtensions(Collections.emptySet());
		exportOptions.setExtensions(Set.of("cgmesControlAreas"));

		if (control.getValue().toString().equals("netInterchange")
		if (control.getValue() != null && control.getValue().toString().equals("netInterchange")

CGMES: remove extension for Control Areas, use IIDM Area #3149

Are you sure you want to change the base?

CGMES: remove extension for Control Areas, use IIDM Area #3149

Conversation

zamarrenolm commented Sep 20, 2024 • edited by So-Fras Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sonarqubecloud bot commented Dec 16, 2024

Quality Gate passed

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rcourtier Dec 18, 2024 • edited Loading

Choose a reason for hiding this comment

zamarrenolm commented Sep 20, 2024 •

edited by So-Fras

Loading

rcourtier Dec 18, 2024 •

edited

Loading