Conversion from CGMES: incorrect current limits

[bhnm-akbari]

• Do you want to request a feature or report a bug?
  Bug

• What is the current behavior?
  When I use itools to convert CGMES files to XIIDM, the permanentLimit value of iidm:currentLimits1/2 for a line differs from the corresponding cim:CurrentLimit.value. Exporting to AMPL format also inherits the same wrong value. This wrong value is indeed the cim:CurrentLimit.value of some other line.

• If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem
  I use convert-network command of itools to convert the TYNDP2020 files provided by ENTSO-E. Interestingly, including different number of input file sets affects the result. For example, including CH and DE files results in:
  <iidm:line id="_56e86b7f-2db3-4001-a3d1-0d879ced8261" ...> <iidm:alias type="CGMES.Terminal1">_d07901a4-8926-49f8-89a4-78f962a3ed08</iidm:alias> <iidm:alias type="CGMES.Terminal2">_dc654cfa-452b-43b8-918e-1be9c5596578</iidm:alias> <iidm:currentLimits1 permanentLimit="1792.0"/> <iidm:currentLimits2 permanentLimit="1792.0"/> </iidm:line>
  Whereas, including CH, DE, AT, FR, and IT results in:
  <iidm:line id="_56e86b7f-2db3-4001-a3d1-0d879ced8261" ...> <iidm:alias type="CGMES.Terminal1">_d07901a4-8926-49f8-89a4-78f962a3ed08</iidm:alias> <iidm:alias type="CGMES.Terminal2">_dc654cfa-452b-43b8-918e-1be9c5596578</iidm:alias> <iidm:currentLimits1 permanentLimit="1792.0"/> <iidm:currentLimits2 permanentLimit="1760.0"/> </iidm:line>

• What is the expected behavior?
  The expected output would be:
  <iidm:line id="_56e86b7f-2db3-4001-a3d1-0d879ced8261" ...> <iidm:alias type="CGMES.Terminal1">_d07901a4-8926-49f8-89a4-78f962a3ed08</iidm:alias> <iidm:alias type="CGMES.Terminal2">_dc654cfa-452b-43b8-918e-1be9c5596578</iidm:alias> <iidm:currentLimits1 permanentLimit="1760.0"/> <iidm:currentLimits2 permanentLimit="1760.0"/> </iidm:line>

• Please tell us about your environment:

  • PowSyBl Version: 1.2.0
  • OS Version: Windows 10

[annetill]

Hi, thanks for reporting this behaviour. Without the file, it is not easy for us to understand but maybe you can try to give us some clues: in which country is this line located? I understand that it is probably a line inside a country (CH or DE), why do you expect that the permanent limit to change to 1760 at both side? Finally we maybe expect that nothing change and stay to 1792? So please if you can give us more details.

[miovd]

Hello,
Could you check when you include AT, FR and IT files that there is no other additional OperationalLimit for _56e86b7f-2db3-4001-a3d1-0d879ced8261 or its terminals in these files? The definition of PATL in our code takes the lowest limit value so it appears weird to me that we ignore a lower limit that the one defined if it is present with only CH and DE.

[bhnm-akbari]

Hello,
Thank you for the replies.
The line lies in DE and the relevant capacity values I found in the input EQ file are the following:
<cim:ACLineSegment rdf:ID="_56e86b7f-2db3-4001-a3d1-0d879ced8261"> ... <fgh:ACLineSegment.imax2>2024.0000000000</fgh:ACLineSegment.imax2> <fgh:ACLineSegment.imax1>1848.0000000000</fgh:ACLineSegment.imax1> <fgh:ACLineSegment.imax3>1760.0000000000</fgh:ACLineSegment.imax3> <fgh:ACLineSegment.imax4>1760.0000000000</fgh:ACLineSegment.imax4> <fgh:ACLineSegment.imax5>1848.0000000000</fgh:ACLineSegment.imax5> <fgh:ACLineSegment.imax6>1760.0000000000</fgh:ACLineSegment.imax6> </cim:ACLineSegment>
and
<cim:CurrentLimit rdf:ID="_776481b7-688c-4372-9f31-cef4ca69ad47"> <cim:IdentifiedObject.name>Normal_Ir</cim:IdentifiedObject.name> <cim:CurrentLimit.value>1760.0000000000</cim:CurrentLimit.value> <cim:OperationalLimit.OperationalLimitSet rdf:resource="#_9a864dc9-4088-4358-914b-1a58f89e9a9e"/> <cim:OperationalLimit.OperationalLimitType rdf:resource="#_ff03ba1d-8217-48f9-8ca7-76204f87e4db"/> </cim:CurrentLimit>
and
<cim:CurrentLimit rdf:ID="_71da32e7-1e31-488c-aeb5-825a97270bc1"> <cim:IdentifiedObject.name>Normal_Ir</cim:IdentifiedObject.name> <cim:CurrentLimit.value>1760.0000000000</cim:CurrentLimit.value> <cim:OperationalLimit.OperationalLimitSet rdf:resource="#_2ac81163-0bdd-4fab-9e6c-94fc30e5c1d8"/> <cim:OperationalLimit.OperationalLimitType rdf:resource="#_ff03ba1d-8217-48f9-8ca7-76204f87e4db"/> </cim:CurrentLimit>
I’m not sure if fgh:ACLineSegment.imax values are considered at all, but the expected capacity would not be 1792 with any consideration.
Side note: I could not convert DE alone because of an error, which I can post if it helps.

[miovd]

Thank you for your answer!
Indeed, imax values are not taken into account by PowSyBl's importer. Could you also send here the OperationalLimitType, there may be a mishap in the way we read it? Knowing the error which prevents you to import DE file alone could also help, thanks!

[bhnm-akbari]

Thank you; here's the OperationalLimitType:
<cim:OperationalLimitType rdf:ID="_ff03ba1d-8217-48f9-8ca7-76204f87e4db"> <cim:IdentifiedObject.name>PATL</cim:IdentifiedObject.name> <cim:OperationalLimitType.direction rdf:resource="http://iec.ch/TC57/2013/CIM-schema-cim16#OperationalLimitDirectionKind.absoluteValue"/> <entsoe:OperationalLimitType.limitType rdf:resource="http://entsoe.eu/CIM/SchemaExtension/3/1#LimitTypeKind.patl"/> </cim:OperationalLimitType>
I'll send the error, once I have access to it (in the evening).

[miovd]

(Sorry, I forgot to also ask you the associated OperationalLimitSet... thank you!)

[bhnm-akbari]

No worries; I'm here to have my problem solved. Here you go:
<cim:OperationalLimitSet rdf:ID="_9a864dc9-4088-4358-914b-1a58f89e9a9e"> <cim:IdentifiedObject.name>Ratings</cim:IdentifiedObject.name> <cim:OperationalLimitSet.Terminal rdf:resource="#_d07901a4-8926-49f8-89a4-78f962a3ed08"/> </cim:OperationalLimitSet>
and
<cim:OperationalLimitSet rdf:ID="_2ac81163-0bdd-4fab-9e6c-94fc30e5c1d8"> <cim:IdentifiedObject.name>Ratings</cim:IdentifiedObject.name> <cim:OperationalLimitSet.Terminal rdf:resource="#_dc654cfa-452b-43b8-918e-1be9c5596578"/> </cim:OperationalLimitSet>

[bhnm-akbari]

Update:
The error in converting the DE files was due to the fact <md:Model.version> is expected to be an integer, but it was a string. After editing this in all files (EQ, SSH, SV, and TP), I managed to convert DE alone.
The result surprisingly matches the expected behavior.
<iidm:line id="_56e86b7f-2db3-4001-a3d1-0d879ced8261" ...> <iidm:alias type="CGMES.Terminal1">_d07901a4-8926-49f8-89a4-78f962a3ed08</iidm:alias> <iidm:alias type="CGMES.Terminal2">_dc654cfa-452b-43b8-918e-1be9c5596578</iidm:alias> <iidm:currentLimits1 permanentLimit="1760.0"/> <iidm:currentLimits2 permanentLimit="1760.0"/>

Still, I wouldn't call the issue resolved, because I need to convert all my files together to account for inter-country connections.
Thank you

[miovd]

Hm, I'm not seeing anything that could be misinterpreted by PowSyBl's importer... Could you please check if in your logs' warnings, you have a message that looks like this : Several permanent limits defined for Terminal _dc654cfa-452b-43b8-918e-1be9c5596578. Only the lowest is kept. or Several permanent limits defined for Terminal _d07901a4-8926-49f8-89a4-78f962a3ed08. Only the lowest is kept. when importing several input files?

[bhnm-akbari]

In CMD, the only output message is
Generating file ...
How should I check the logs?

[miovd]

Ah yes, you may need to go to the /etc/ folder of your distribution and change the logback-itools.xml file to:

<!--

    Copyright (c) 2016, All partners of the iTesla project (http://www.itesla-project.eu/consortium)
    This Source Code Form is subject to the terms of the Mozilla Public
    License, v. 2.0. If a copy of the MPL was not distributed with this
    file, You can obtain one at http://mozilla.org/MPL/2.0/.

-->
<!--Configuration of the command line process log-->
<configuration>
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <Pattern>%d{yyyy-MM-dd_HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</Pattern>
        </encoder>
    </appender>
    <root level="WARN">
        <appender-ref ref="STDOUT" />
    </root>
</configuration>
`

[bhnm-akbari]

There are indeed a number of the following warning type:
WARN com.powsybl.cgmes.conversion.Context - Fixed Permanent Limit. Reason: Several permanent limits defined for Terminal ... . Only the lowest is kept.
But none is related to the adjacent terminals of the line we're interested in.

[bhnm-akbari]

May I ask if you're working on this issue? Otherwise, I'll have to consider alternative approaches.

I have anyway to go through some other conversion steps to convert AMPL format to MATPOWER format.

Thank you.

[miovd]

Sorry for the delayed answer! I tried looking a bit around the code but I really can't seem to find where it could go wrong and it is a bit difficult to try and guess without files to test... @zamarrenolm do you have any idea on why the CGMES importer has this behavior?

[annetill]

If you want us to help, you have to provide us a way to reproduce your issue. The issue could be in our project, but it could be also in your files. We know the code, but we don't have your files. If you don't provide this way to reproduce the behavior, we will not work on it. And yes, if you prefer, use another tool.

[bhnm-akbari]

I'm not allowed to share the files. That's why I haven't done it so far. I've detected some clues to the issue. I will share my findings when finalized. If still needed, I will consider how to share some data without violating the license terms.

[bhnm-akbari]

I sketched the network diagram based on CIM input files.

The line that was referred to is Line1 which is expected to have capacity of 1760. It is connected to another line, namely Line2 which is expected to have a capacity of 1792. Both lines lie in DE.

When converting only DE, everything goes as expected: Line1 is converted to a iidm:line element with a capacity of 1760 and Line2 is converted to a iidm:danglingLine element with a capacity of 1792.

However, when I convert DE+CH, Line2, Line3, and Line4 do not appear in the xiidm file and the capacity of Line2 (i.e., 1792) is assigned to the capacity of Line1. So the problem is not limited to an incorrect capacity assignment, but also includes removal of some line elements.

My understanding is that the issue arises from the fact that there are more than two lines connected to a topological node (TN) at the boundary. However, the tieLine element in xiidm seems to support only two lines connected to a topological node at the boundary.

If you confirm my understanding, a solution would be to avoid merging lines at the boundary to a single tie line, but rather keep them as separate lines.

[mathbagu]

Thank you for these new elements. @annetill @MioRtia I think we can have access to this file asking people from the RTE R&D department. They probably already have this file or can download it from the ENTSO-E.

If I understand well your sketch, it seems that Line3 and Line4 are also dangling lines, right? We have here three danglingLines connected to the same TN. This use case is not supported by powsybl (I don't remember having encountered it in the past). In powsybl, we expect exactly two "matching" boundaryLine in order to create TieLine. If there are more, you should see an error message in the logs:

Invalid XXX. Reason: Too many equipment at boundary node

@bhnm-akbari Could you confirm?

My understanding is that there is probably no issue in the currentLimit conversion, except maybe the fact that when several limits are defined in the CGMES file, we keep only the lowest to be conservative. We do not want to miss overloads that could lead to security issues in our operational process.

If can have access to this file quickly, we should be able to diagnose more precisely the issue you described. Without it, it's really difficult for us. Is it possible to send us the logs (debug level) of the conversion tool by email at mathieu.bague AT rte-france.com?

[bhnm-akbari]

Thank you for your answer.

The data set is called TYNDP 2020 that can be requested from the following page: https://docstore.entsoe.eu/stum/

When converting only DE, everything goes as expected: Line1 is converted to a iidm:line element with a capacity of 1760 and Line2 is converted to a iidm:danglingLine element with a capacity of 1792.

When I convert DE+CH, Line2, Line3, and Line4 do not appear in the xiidm file and the capacity of Line2 (i.e., 1792) is assigned to the capacity of Line1. So the problem is not limited to an incorrect capacity assignment, but also includes removal of some line elements.

When I convert CH alone, Line3 and Line4 are merged together to form a tieLine, although they both lie in CH! So they are not converted to danglingLines.

I confirm the error. The following message is given when converting CH and DE together:

Invalid _0a7ff20b9b3e464a8f6dfbead0dbcb6d. Reason: Too many equipment at boundary node

Note that the current limit of one line is assigned to another line. Additionally, the assigned value is not the minimum of the two current limits.

I will send the log to the mentioned email address.

[mathbagu]

Thanks for the logs.

As explained this morning, this is a totally new use case we never encountered, but with your additional information we understand better the issue. We'll try to find a way to solve the issue you are facing.

[zamarrenolm]

We are also working on creating a minimal example that can be shared and reflects the explained configuration, to verify the incorrect limit assignment.

As @mathbagu said, three lines lying at the same boundary node is not supported by the current version. It is not described in the allowed ENTSO-E configurations that can be found at boundaries. Is there a particular reason for modelling that point in this way ?

[bhnm-akbari]

Thank you.

What about the case where the topological node with more than two connected ACLineSegments lies within the boundary not at the boundary? Is this case currently supported?

A possible application of such a configuration would be to model Tee-off line connections

[zamarrenolm]

I have been able to reproduce your issue with a minimal case built manually. I will analyze it and share it with you.

About modelling tee-of connections: as soon as the boundary nodes (published in ENTSO-E boundary files) receive at most two branches, everything should be ok.

[bhnm-akbari]

@zamarrenolm That's great; I'll be thrilled to follow the developments in this regard.

By definition at a Tee-off connection point, three line segments meet. But this is exactly what you you're addressing right now, and based on your explanation I assume this is not an issue if such a configuration exists somewhere not adjacent to the boundary.

[zamarrenolm]

Yes, I understand that tee-off connection modelling requires three line segments meeting at the same node.

The only issue is that this node can not be a boundary, only two "sides" are expected at a boundary in ENTSO-E definitions, with only one equipment at modelled at each side. As reference, the ENTSO-E extensions to CGMES (http://entsoe.eu/CIM/SchemaExtension/3/1#) define a naming scheme with allowing only "fromEnd" and "toEnd" equipment. The description of the "fromEndName" attribute:

The attribute is used for an exchange of a human readable name with length of the string 32 characters maximum. The attribute covers two cases:

• if the Boundary point is placed on a tie-line the attribute is used for exchange of the geographical name of the substation to which the “From” side of the tie-line is connected to.
• if the Boundary point is placed in a substation the attribute is used for exchange of the name of the element (e.g. PowerTransformer, ACLineSegment, Switch, etc) to which the “From” side of the Boundary point is connected to.
  The attribute is required for the Boundary Model Authority Set where it is used only for the TopologicalNode in the Boundary Topology profile and ConnectivityNode in the Boundary Equipment profile.

We will provide an example of how to keep the modelling the tee-off connection while preserving the restriction of having only two equipment at the boundary point.

[bhnm-akbari]

Thank you for the information. There are at least five instances of such connections at the boundary of TYNDP 2020 files, leading to the following warning during import:
Invalid ... Reason: Too many equipment at boundary node

[zamarrenolm]

Please find attached two sets of files that reproduce and fix the problem in a minimal test network.
They follow the identifiers you used in your drawing: line1 and line2 on DE side, line3 and line4 on CH side.
Also, I have applied the same identifiers and values for the limits of line1 and line2 in DE.

In the file minimal-reproduce-error.zip the error reported is reproduced: if only DE files are loaded the limit for line1 is 1760. When all files, from both DE and CH are imported, the limit for line1 is incorrectly set as 1792. The following diagram shows the elements:

The topological node at the boundary, TN1_BD, is connected to line2, line3 and line4. As you can see, CIMdesk is not drawing explicitly the three equipment connected at boundary node, although the references are correct.

The proposal to fix the issue is to model a zero-impedance line between the boundary and the point where line3 and line4 are connected, that will be tee-off connection point and will be kept inside CH. The files proposed for the fix can be found in this file: minimal-fix-with-z0.zip. The diagram that shows the elements in this case is:

With the proposed fix we keep only two branches at the boundary, and the line2 maintains its limit of 1760.

[bhnm-akbari]

Thank you for the test network and the associated fixed variant. It is indeed a wise solution. Do you plan to automate the fix in a near-future release or is the user expected to manually modify the input CGMES files before importing into PowSyBl?

[zamarrenolm]

Hope this helps in your analysis.
For the moment we have not planned to automate the fix in the software, but we will discuss it with the rest of the team.