imports loaded in package.mo not found when trying to load a model #199

martinscripts · 2024-02-29T13:57:00Z

Description

This may be connected to #133.
I am trying to load a model which uses the dynawo library
There are several packages loaded inside the package.mo of dynawo:

encapsulated package Dynawo "Dynawo models library"
  // generic import commands
  import Complex "Complex numbers foundation class";
  import Modelica;
  import Modelica.ComplexMath "Complex numbers operators (+, - , *, exp, abs...)";
  import Modelica.Icons;
  import Dynawo;
  import Dynawo.AdditionalIcons;
  import Dynawo.Electrical.SystemBase;
  import Dynawo.Types "Standard Dynawo variable types";

which can't be found when loading a model through ModelicaSystem() in OMPython.

Maybe I am just missing something?

Thank you in advance :)

Steps to reproduce

from OMPython import ModelicaSystem
DYNAWO_PACKAGE_PATH = "[path-to-dynawo]/dynawo/sources/Models/Modelica/Dynawo/package.mo"
mod = ModelicaSystem(DYNAWO_PACKAGE_PATH, "Dynawo.Examples.SMIB.SMIBStepPm", ["Modelica"])

returns the error Error: Function ComplexMath.'abs' not found in scope GeneratorSynchronous.

Expected behavior

It should be loaded properly because ComplexMath is loaded in dynawo's package.mo, as pasted above.
And in OMEdit it runs.

writing mod = ModelicaSystem(DYNAWO_PACKAGE_PATH, "Dynawo.Examples.SMIB.SMIBStepPm", ["Modelica", "Modelica.ComplexMath"]) did not change it.

Version and OS

Python Version 3.10.11
OMPython Version 3.3.0
OpenModelica Version 1.18.0
OS: Windows 10, 64 bit

The text was updated successfully, but these errors were encountered:

adeas31 · 2024-02-29T14:11:22Z

This should work fine.
What happens if you run this script with omc.exe from the command line?

loadFile("[path-to-dynawo]/dynawo/sources/Models/Modelica/Dynawo/package.mo");
getErrorString();
simulate(Dynawo.Examples.SMIB.SMIBStepPm);
getErrorString();

arun3688 · 2024-02-29T15:05:51Z

@martinscripts when i tried to load this model in OMEdit, i get the following error

[3] 15:54:14 Translation Error
[Dynawo.Electrical.Controls.Basics.SwitchOff: 19:3-19:48]: [Import Dynawo.NonElectrical.Logs.TimelineKeys not found in scope <top>.](omeditmessagesbrowser:///Dynawo.Electrical.Controls.Basics.SwitchOff?lineNumber=19)

[4] 15:54:14 Translation Error
[Dynawo.Electrical.Controls.Basics.SwitchOff.SwitchOffLine: 29:5-29:46]: [Variable TimelineKeys.LineOpen not found in scope SwitchOffLine.](omeditmessagesbrowser:///Dynawo.Electrical.Controls.Basics.SwitchOff.SwitchOffLine?lineNumber=29)

And i cannot find the model TimeLineKeys.mo in the package, could you please make sure the model TimeLineKeys.mo exists

martinscripts · 2024-03-01T11:52:05Z

@adeas31
running with omc.exe seems to work fine. I get a bunch of warnings but no error.

here the first part of the output:

Microsoft Windows [Version 10.0.19045.4046]
(c) Microsoft Corporation. Alle Rechte vorbehalten.

C:\Users\mfranke>"C:\OpenModelica\bin\omc.exe" "D:\User\mfranke\documents\projects\colib\src\GovCT2\run_model.mos"
true
"Notification: Automatically loaded package Modelica 3.2.3 due to uses annotation.
Notification: Automatically loaded package Complex 3.2.3 due to uses annotation.
Notification: Automatically loaded package ModelicaServices 3.2.3 due to uses annotation.
"
record SimulationResult
    resultFile = "C:/Users/mfranke/Dynawo.Examples.SMIB.SMIBStepPm_res.mat",
    simulationOptions = "startTime = 0.0, stopTime = 30.0, numberOfIntervals = 500, tolerance = 1e-06, method = 'dassl', fileNamePrefix = 'Dynawo.Examples.SMIB.SMIBStepPm', options = '', outputFormat = 'mat', variableFilter = '.*', cflags = '', simflags = '-initialStepSize=\\'0.001\\' -lv=\\'LOG_STATS\\' -nls=\\'kinsol\\' -s=\\'ida\\' -nlsLS=\\'klu\\' -maxIntegrationOrder=\\'2\\' -maxStepSize=\\'10\\' -emit_protected'",
    messages = "LOG_SUCCESS       | info    | The initialization finished successfully without homotopy method.
LOG_STATS         | info    | ### STATISTICS ###
|                 | |       | | timer
|                 | |       | | |    0.0050984s          reading init.xml
|                 | |       | | |    0.0007751s          reading info.xml
|                 | |       | | |    0.0010354s [  1.5%] pre-initialization
|                 | |       | | |    0.0004787s [  0.7%] initialization
|                 | |       | | |     2.66e-05s [  0.0%] steps
|                 | |       | | |     0.001309s [  1.9%] solver (excl. callbacks)
|                 | |       | | |    0.0058192s [  8.4%] creating output-file
|                 | |       | | |     3.15e-05s [  0.0%] event-handling
|                 | |       | | |    0.0010795s [  1.6%] overhead
|                 | |       | | |     0.059764s [ 85.9%] simulation
|                 | |       | | |    0.0695439s [100.0%] total
|                 | |       | | events
|                 | |       | | |     1 state events
|                 | |       | | |     0 time events
|                 | |       | | solver: ida
|                 | |       | | |   810 steps taken
|                 | |       | | |  1055 calls of functionODE
|                 | |       | | |    67 evaluations of jacobian
|                 | |       | | |    16 error test failures
|                 | |       | | |     0 convergence test failures
|                 | |       | | | 0.0119926s time of jacobian evaluation
LOG_SUCCESS       | info    | The simulation finished successfully.
",
    timeFrontend = 0.2678431,
    timeBackend = 0.2779029,
    timeSimCode = 0.0340495,
    timeTemplates = 0.0383436,
    timeCompile = 8.264896800000001,
    timeSimulation = 6.842613,
    timeTotal = 15.7369697
end SimulationResult;

martinscripts · 2024-03-01T11:57:54Z

@arun3688
sorry, I forgot. This is a problem on the dynawo side, I think. The timeline keys files are in the gitignore, so not in the repo, but they're necessary, at least on windows OS.

copy the attached files into the folder dynawo\dynawo\sources\Models\Modelica\Dynawo\NonElectrical\Logs in your local dynawo repo, then it should work.
colib-main-windows_files (1).zip
(uploading .mo files wasnt possible, so I had to upload them in a .zip)

martinscripts · 2024-03-06T14:08:39Z

Is there anything else I can supply in order to answer my question? :)

adeas31 · 2024-03-06T14:19:59Z

I just talked with @arun3688 and he will try to investigate this further.

In the meantime can you try the latest version of OpenModelica. The version 1.18 is almost 2-3 yrs old. Also try updating the OMPython to the latest version as well.

martinscripts · 2024-03-06T14:39:06Z

Thank you. Unfortunately, the OM 1.18 is a dependency of Dynawo, the library I am using.

arun3688 · 2024-03-06T15:09:21Z

@martinscripts I tested the library in OMPython after putting the missing files in dynawo\dynawo\sources\Models\Modelica\Dynawo\NonElectrical\Logs and it worked perfectly fine see below

Python 3.8.5 (default, Sep  3 2020, 21:29:08) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from OMPython import ModelicaSystem
>>> mod = ModelicaSystem("C:/OPENMODELICAGIT/dynawo/dynawo/sources/Models/Modelica/Dynawo/package.mo", "Dynawo.Examples.SMIB.SMIBStepPm", lmodel=["Modelica"])
>>> mod.simulate()
LOG_SUCCESS       | info    | The initialization finished successfully without homotopy method.
LOG_SUCCESS       | info    | The simulation finished successfully.

The Dynawo library works with recent OpenModelica version v1.23.0-dev-358-g07765ef93c-cmake , So i suggest you to download the latest OpenModelica version and also update the OMPython from pip and test it and it should work by default

martinscripts · 2024-03-06T15:12:46Z

Thanks so much for your help.

while it may work, the version it is a requirement for contributing to dynawo.
I will update OMPython and hope that it works. If not, I'll see how to deal with the OM version.

Cheers! :)
.Martin

martinscripts · 2024-03-06T16:26:36Z

Updating OMPython got me a step further. The problem that I described doesn't happen anymore. Thank you!

martinscripts · 2024-03-07T09:23:41Z

Hello again,

I ran into a similar problem again once I tried to load my own model (outside of Dynawo) with Dynawo as a dependency:

from OMPython import ModelicaSystem
MODEL_PACKAGE_PATH = "[path-to-own-project]/src/KundurTwoArea/package.mo"
DYNAWO_PACKAGE_PATH = "[path-to-dynawo]/dynawo/sources/Models/Modelica/Dynawo/package.mo"
mod = ModelicaSystem(MODEL_PACKAGE_PATH, "KundurTwoArea.Grid.LoadFlow", ["Modelica", DYNAWO_PACKAGE_PATH])

Is this correct calling of ModelicaSystem()?

Again, it does not find the following packages:

[[...]/src/KundurTwoArea/package.mo:21:3-21:24:writable] Error: Import Modelica.Icons not found in scope <top>.
[[...]/src/KundurTwoArea/package.mo:27:3-27:24:writable] Error: Base class Icons.Package not found in scope KundurTwoArea.
Error: Class KundurTwoArea.Grid.LoadFlow not found in scope <top>.

I want to try OMEdit 1.22.2, I installed it as a second instance alongside 1.18.0. (see here)
Is there a way to tell OMPython which version to use? (or to check which one it is using?)

Best regards,
.Martin

adeas31 · 2024-03-07T09:28:38Z

I think OMPython uses the OPENMODELICAHOME environment variable. Set the variable to the version you want to use.

martinscripts · 2024-03-07T09:37:01Z

Ok, it is set to 1.22.2.
So with this version it does not work.

Any further ideas?
The imports still happen in the package.mo like above, but in my own model this time.
They don't seem to be found. Maybe I am missing something here 🙈 .

casella · 2024-03-07T10:00:38Z

Thank you. Unfortunately, the OM 1.18 is a dependency of Dynawo, the library I am using.

@martinscripts I am not sure I understand what you mean here. The Dynawo software, which is a hybrid C++/Modelica thing, may have a dependency on a specific version of OpenModelica for quality assurance reasons. But if you just use the Dynawo Modelica library, which I understand is found here, the only dependency is on the Modelica Standard Library 3.2.3, which is automatically installed when you install OpenModelica.

We understand's RTE's requirements, as they are using Dynawo in production they have to make sure all the dependencies are well tested and are a bit reluctant at updating the OpenModelica version. On the other hand, OpenModelica has been vastly improved since version 1.18.0, so we can't really offer assistance to you if you stick to a version of the software that lacks two an half years of bug fixes and developments. Unless there is a very specific reason for that.

martinscripts · 2024-03-07T10:07:10Z

Hey @casella, at the moment I am working with and referring to OMEdit 1.22.2, which is the latest official release.
So no need to worry abut 1.18 at the moment.

While the update of OMPython helped with the problem I originally described, it did not work with an own package outside the dynawo library.
Using 1.22.2 didn't solve it either, unfortunately.

casella · 2024-03-07T10:10:46Z

OK. Can you please provide @arun3688 an updated libray with the keys, so he can try to reproduce you problem and fix it?

adeas31 · 2024-03-07T10:13:35Z

I am guessing it fails to load the dynamo library. Is there any other error message besides this?

martinscripts · 2024-03-07T10:18:48Z

@adeas31 no only those messages.

@casella @arun3688 the dynawo library did not change, see above for where to put the timelineKeys files.

Here is the model I am trying to load, which in turn uses dynawo.
KundurTwoArea.zip

adeas31 · 2024-03-07T10:21:02Z

Can you post the output of the following script,

getVersion();
getErrorString();
loadModel(Modelica);
getErrorString();
loadFile("[path-to-dynawo]/dynawo/sources/Models/Modelica/Dynawo/package.mo");
getErrorString();
loadFile("[path-to-own-project]/src/KundurTwoArea/package.mo");
getErrorString();
simulate(KundurTwoArea.Grid.LoadFlow);
getErrorString();

casella · 2024-03-07T10:22:33Z

@casella @arun3688 the dynawo library did not change, see above for where to put the timelineKeys files.

Can you please open a ticket on the dynawo project where you ask the developers to remove the timelineKeys files from .gitignore and manage them properly on the git repository? We will soon add Dynawo to the OMC package manager, and this is essential for the whole thing to work properly in all cases. Thanks!

martinscripts · 2024-03-07T10:39:19Z

@adeas31

"OpenModelica v1.22.2 (64-bit)"
""
true
""
true
"Warning: Dynawo requested package Modelica of version 3.2.3. Modelica 4.0.0 is used instead which states that it is only compatible with a conversion script. Use convertPackageToLibrary(Dynawo, Modelica, \"4.0.0\") to run the conversion script or proceed with potential issues as a result.
"
true
""
record SimulationResult
    resultFile = "",
    simulationOptions = "startTime = 0.0, stopTime = 1.0, numberOfIntervals = 500, tolerance = 1e-06, method = 'dassl', fileNamePrefix = 'KundurTwoArea.Grid.LoadFlow', options = '', outputFormat = 'mat', variableFilter = '.*', cflags = '', simflags = ''",
    messages = "Failed to build model: KundurTwoArea.Grid.LoadFlow",
    timeFrontend = 0.0,
    timeBackend = 0.0,
    timeSimCode = 0.0,
    timeTemplates = 0.0,
    timeCompile = 0.0,
    timeSimulation = 0.0,
    timeTotal = 0.0
end SimulationResult;
"Warning: Dynawo requested package Modelica of version 3.2.3. Modelica 4.0.0 is used instead which states that it is only compatible with a conversion script. Use convertPackageToLibrary(Dynawo, Modelica, \"4.0.0\") to run the conversion script or proceed with potential issues as a result.
[D:/User/mfranke/documents/projects/dynawo_fork/dynawo/sources/Models/Modelica/Dynawo/Electrical/Buses/Bus.mo:30:3-30:38:writable] Error: Function ComplexMath.'abs' not found in scope Bus.
"

casella · 2024-03-07T10:47:31Z

"Warning: Dynawo requested package Modelica of version 3.2.3. Modelica 4.0.0 is used instead which states that it is only compatible with a conversion script. Use convertPackageToLibrary(Dynawo, Modelica, "4.0.0") to run the conversion script or proceed with potential issues as a result.

I suspected this could be the problem, and in fact here it is. Unfortunately, Dynawo is still using MSL 3.2.3, if OMC loads MSL 4.0.0, nothing will work. The question is, why isn't the uses annotation handled properly?

adeas31 · 2024-03-07T10:48:22Z

Now it makes sense. You should get this error message from OMPython as well. @arun3688 can you fix OMPython so it shows the error messages properly.

[D:/User/mfranke/documents/projects/dynawo_fork/dynawo/sources/Models/Modelica/Dynawo/Electrical/Buses/Bus.mo:30:3-30:38:writable] Error: Function ComplexMath.'abs' not found in scope Bus.
Seems Dynawo is not compatible with Modelica 4.0.0

The following script should work without error,

loadFile("[path-to-dynawo]/dynawo/sources/Models/Modelica/Dynawo/package.mo");
getErrorString();
loadFile("[path-to-own-project]/src/KundurTwoArea/package.mo");
getErrorString();
simulate(KundurTwoArea.Grid.LoadFlow);
getErrorString();

And the following OMPython code should also work,

from OMPython import ModelicaSystem
MODEL_PACKAGE_PATH = "[path-to-own-project]/src/KundurTwoArea/package.mo"
DYNAWO_PACKAGE_PATH = "[path-to-dynawo]/dynawo/sources/Models/Modelica/Dynawo/package.mo"
mod = ModelicaSystem(MODEL_PACKAGE_PATH, "KundurTwoArea.Grid.LoadFlow", [("Modelica","3.2.3"), DYNAWO_PACKAGE_PATH])

adeas31 · 2024-03-07T10:49:13Z

"Warning: Dynawo requested package Modelica of version 3.2.3. Modelica 4.0.0 is used instead which states that it is only compatible with a conversion script. Use convertPackageToLibrary(Dynawo, Modelica, "4.0.0") to run the conversion script or proceed with potential issues as a result.

I suspected this could be the problem, and in fact here it is. Unfortunately, Dynawo is still using MSL 3.2.3, if OMC loads MSL 4.0.0, nothing will work. The question is, why isn't the uses annotation handled properly?

Uses annotation is handled. The script explicitly loads Modelica 4.0.0 before loading the dynawo library. I updated the script.

arun3688 · 2024-03-07T10:49:42Z

@martinscripts I found the issue, the dependency package are not loaded in OMPython and that is causing the issue in OMPython

martinscripts · 2024-03-07T11:01:32Z

thanks people, That's a really promising lead!

@adeas31
the .mos script works without error now.
The OMPython script doesn't work, still. It throws the same errors as before.
(this is how I changed it:)
mod = ModelicaSystem(MODEL_PACKAGE_PATH, "KundurTwoArea.Grid.LoadFlow", [("Modelica", "3.2.3"), DYNAWO_PACKAGE_PATH])

adeas31 · 2024-03-07T11:02:41Z

The OMPython script doesn't work, still. It throws the same errors as before.

Yeah. Me and @arun3688 spotted the error in OMPython. @arun3688 will make a fix and will make a new pip package.

casella · 2024-03-07T11:21:41Z

@martinscripts in general I would try to avoid explicitly loading installed libraries. Every library should have proper uses annotations, so that the package manager can figure this out automatically.

martinscripts · 2024-03-07T12:01:56Z

okay, thanks for the help.

Every library should have proper uses annotations

Would this look something like this, in the package.mo that I was loading?

within ;

package KundurTwoArea "Kundur Two Area test system used for small signal stability studies"
 
    // generic import commands
  import Complex "Complex numbers foundation class";
  import Modelica;
  import Modelica.ComplexMath "Complex numbers operators (+, - , *, exp, abs...)";
  import Modelica.Icons;
  import Dynawo;
  import Dynawo.AdditionalIcons;
  import Dynawo.Electrical.SystemBase;
  import Dynawo.Types "Standard Dynawo variable types";

  extends Icons.Package;
  
  annotation(
    uses(Modelica(version = "3.2.3")),
    preferredView = "info",
    Documentation(info = "<html><head></head><body>This package contains the two-area test system described in Example 12.6 of Prabha Kundur's Power System Stability and Control.</body></html>"));
end KundurTwoArea;

casella · 2024-03-07T12:56:13Z

Yes. Note that if you drag-and-drop a model from a library into another model, possibly contained in a package, OMEdit automatically adds the appropriate uses annotation, so that the library is automatically loaded when loading the model or package.

As a matter of fact, I'm not sure why you add those import statements. If you build models graphically, there is no need for them. You just need to have the libraries loaded. This happens automatically if

they have been installed with the package manager
there is a uses annotation for each of them

Note that Modelica Standard Library is automatically installed when you install OpenModelica on Windows. Dynawo is currently not included in the package manager, so you need to download it somewhere and open it manually every time you start OMEdit - you can do that with one click in the Welcome page using the recent file list.

Until we include Dynawo in the package manager, to get it automatically loaded you need to add a uses(Dynawo) annotation to your package and manually copy the Dynawo package directory in your %appdata%\.openmodelica\libraries directory.

casella · 2024-03-07T12:57:41Z

What I mean is that import XXX.YYY; doesn't mean "load the library XXX.YYY", it just means that you can use YYY instead of XXX.YYY to access it in your models. To load them, either open them manually or rely on the uses annotation.

arun3688 · 2024-03-07T13:26:12Z

@martinscripts the issue is fixed with this commit 1fcbcac, i have published the package to pip, and you can do pip install OMPython and start testing it. I tested your package and it works now

>>> from OMPython import ModelicaSystem
>>> mod = ModelicaSystem("C:/OPENMODELICAGIT/KundurTwoArea/package.mo", "KundurTwoArea.Grid.LoadFlow", lmodel=["C:/OPENMODELICAGIT/dynawo/dynawo/sources/Models/Modelica/Dynawo/package.mo"], verbose=False)
>>> mod.simulate()
LOG_SUCCESS       | info    | The initialization finished successfully without homotopy method.
LOG_SUCCESS       | info    | The simulation finished successfully.

casella · 2024-03-07T13:28:01Z

@martinscripts please close the ticket if this now works for you.

martinscripts · 2024-03-07T13:42:06Z

@casella I'm not fully sure where you would add used(Dynawo), until now it seems like I didn't need that (or I overlooked it).

Thank you all for your help. The model builds now, seems to work fine :).
Best regards,
.Martin

casella · 2024-03-07T14:22:13Z

@casella I'm not fully sure where you would add used(Dynawo),

Once we have put Dynawo in the package manager, you should put this annotation

annotation(uses(Modelica(version = 3.2.3), Dynawo), ...);

at the end of your KundurTwoArea package. This would ensure that as soon as you open KundurTwoArea, the right version of the Modelica Standard library and the latest installed version of Dynawo will be automatically loaded.

martinscripts closed this as completed Mar 6, 2024

martinscripts reopened this Mar 7, 2024

martinscripts mentioned this issue Mar 7, 2024

Missing TimelineKeys Class dynawo/dynawo#1077

Closed

arun3688 mentioned this issue Mar 7, 2024

load the dependent libraries using ModelicaSystem #201

Merged

martinscripts closed this as completed Mar 7, 2024

imports loaded in package.mo not found when trying to load a model #199

imports loaded in package.mo not found when trying to load a model #199

Comments

martinscripts commented Feb 29, 2024

Description

Steps to reproduce

Expected behavior

Version and OS

adeas31 commented Feb 29, 2024

arun3688 commented Feb 29, 2024

martinscripts commented Mar 1, 2024

martinscripts commented Mar 1, 2024

martinscripts commented Mar 6, 2024

adeas31 commented Mar 6, 2024

martinscripts commented Mar 6, 2024

arun3688 commented Mar 6, 2024

martinscripts commented Mar 6, 2024

martinscripts commented Mar 6, 2024

martinscripts commented Mar 7, 2024 • edited Loading

adeas31 commented Mar 7, 2024

martinscripts commented Mar 7, 2024 • edited Loading

casella commented Mar 7, 2024

martinscripts commented Mar 7, 2024

casella commented Mar 7, 2024

adeas31 commented Mar 7, 2024

martinscripts commented Mar 7, 2024

adeas31 commented Mar 7, 2024

casella commented Mar 7, 2024

martinscripts commented Mar 7, 2024

casella commented Mar 7, 2024 • edited Loading

adeas31 commented Mar 7, 2024

adeas31 commented Mar 7, 2024

arun3688 commented Mar 7, 2024

martinscripts commented Mar 7, 2024 • edited Loading

adeas31 commented Mar 7, 2024

casella commented Mar 7, 2024

martinscripts commented Mar 7, 2024 • edited by casella Loading

casella commented Mar 7, 2024

casella commented Mar 7, 2024 • edited Loading

arun3688 commented Mar 7, 2024

casella commented Mar 7, 2024

martinscripts commented Mar 7, 2024

casella commented Mar 7, 2024 • edited Loading

martinscripts commented Mar 7, 2024 •

edited

Loading

martinscripts commented Mar 7, 2024 •

edited

Loading

casella commented Mar 7, 2024 •

edited

Loading

martinscripts commented Mar 7, 2024 •

edited

Loading

martinscripts commented Mar 7, 2024 •

edited by casella

Loading

casella commented Mar 7, 2024 •

edited

Loading

casella commented Mar 7, 2024 •

edited

Loading