ixmp.Platform() OutOfMemoryError

[Aprillmylmy]

Dear Developer,

I have been using Message properly for half a year, and recently got a connection error of ixmp.Platform().

It shows Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Python Reference Queue", as shown also from the screenshot below. The connection takes a long time and then runs into OutOfMemoryError.

Any idea I could debug? I have re-installed everything but not helpful.

Thanks very much,
Emy

[khaeru]

• Please give the output of message-ix show-versions.
• Describe your system. How much memory do you have available? Were you running any memory-hungry programs at the same time?
• What is in your ixmp configuration file (see the documentation)? What contents do you have in the platform/database named "default"?

Nothing really happens when simply creating the Platform object, i.e. very little additional memory should be needed. Some possibilities include that your system has very little memory (or free memory), or the version of Java you are using somehow uses a lot when it starts up.

[Aprillmylmy]

Thank you Paul;

message-ix show-versions gives below info, which looks fine;
ixmp: 3.5.0
message_ix: 3.5.0
message_ix_models: None
message_data: None

click: 8.1.3
dask: 2022.05.2
genno: installed
graphviz: None
jpype: 1.4.0
… JVM path: /Users/emily/opt/anaconda3/envs/message_env/lib/jli/libjli.dylib
openpyxl: 3.0.9
pandas: 1.4.2
pint: 0.19.2
xarray: 2022.3.0
yaml: 6.0

iam_units: installed
jupyter: installed
matplotlib: 3.5.2
plotnine: 0.8.0
pyam: 1.3.1

GAMS: 39.1.1

python: 3.10.4 | packaged by conda-forge | (main, Mar 24 2022, 17:43:32) [Clang 12.0.1 ]
python-bits: 64
OS: Darwin
OS-release: 19.6.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_AU.UTF-8
LOCALE: ('en_AU', 'UTF-8')

My system is based on an 8 GB mac os operation; I tried to shut down all other apps when running message, but it doesn't work; From the activity monitor, I can see python 10 is called, then rapidly occupies ~2.5 GB memory and then the out of memory error occurs.

For my configurations, I copied the file below - they are the default settings.

class Config:
"""Configuration for ixmp.

For most purposes, there is only one instance of this class, available at
:data:`.ixmp.config` and automatically :meth:`read` from the ixmp configuration
file at the moment the package is imported. (:meth:`save` writes the current values
to file.)

Config is a key-value store. Key names are strings; each key has values of a fixed
type. Individual keys can be accessed with :meth:`get` and :meth:`set`, or by
accessing the :attr:`values` attribute.

Spaces in names are automatically replaced with underscores, e.g. "my key" is
stored as "my_key", but may be set and retrieved as "my key".

Downstream packages (e.g. :mod:`message_ix`, :mod:`message_ix_models`) may
:meth:`register` additional keys to be stored in and read from the ixmp
configuration file.

The default configuration (restored by :meth:`clear`) is:

.. code-block:: json

   {
     "platform": {
       "default": "local",
       "local": {
         "class": "jdbc",
         "driver": "hsqldb",
         "path": "~/.local/share/ixmp/localdb/default"
       },
   }

.. autosummary::
   clear
   get
   keys
   read
   save
   set
   register
   unregister
   add_platform
   get_platform_info
   remove_platform

Parameters
----------
read : bool
    Read ``config.json`` on startup.
"""

#: Fully-resolved path of the ``config.json`` file.
path: Optional[Path] = None

#: Configuration values. These can be accessed using Python item access syntax, e.g.
#: ``ixmp.config.values["platform"]["platform name"]…``.
values: BaseValues

_ValuesClass: Type

[khaeru]

Thanks for this information. Per my question:

What contents do you have in the platform/database named "default"?

…you did not give an answer, but my colleague @OFR-IIASA reports seeing this error message from other users when the local/HyperSQL database is very large. (This makes it another form of iiasa/ixmp#433.) If that is the case on your system, you could start with a new platform/database.

[Aprillmylmy]

Thanks very much, Paul;

Sorry I am not very much familiar with ixmp functions;

How could I extract default database locations/names? and how to start a new platform/database?

Many thanks,
Emily

[khaeru]

These things are explained in the documentation: https://docs.messageix.org/projects/ixmp/en/stable/api.html#configuration —please always check there first.

[Aprillmylmy]

Thanks Paul, this is now fixed after changing to a new local database.