dlopen failed: cannot locate symbol "JNI_OnLoad_awt" referenced by libsubstrate.so

[robert-picking]

Describe the issue
When attempting to build, package and install for android, the build and install succeeds, but when the app is launched for the first time it crashes with the error shown below java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "JNI_OnLoad_awt" referenced by "/data/app/...-sc5Osp1K9aHqgnWO_iRKQw==/lib/arm64/libsubstrate.so"...

This is not an issue when building for a different platform, as building and running a windows exe works just fine and does not crash on startup.

My initial research turned up this bug report that leads me to believe it might be an issue specific to a linux build environment.

Steps to reproduce the issue
I'm not 100% sure what is causing this issue, my first thought would be that is something related to the build environment (linux) as this is not an issue building the same application on windows. This fatal exception occurs when building on both debian and ubuntu.

Describe GraalVM and your environment:

• GraalVM version: graalvm-svm-linux-gluon-21.1.0-dev
• JDK major version: 11
• target platform: android 10 device
• OS: Both ubuntu 20.04 LTS and debian 10 running on WSL2
• Architecture: x86_64

This is using the gluon maven client plugin to build a native image.

• Gluon client-maven-plugin: 0.1.39

More details

[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] V/GraalActivity(30675): onRestart
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] V/GraalActivity(30675): onStart
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] V/GraalActivity(30675): onStart done
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] V/GraalActivity(30675): onResume
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] V/GraalActivity(30675): onResume done
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] V/GraalActivity(30675): surfaceCreated for com.gluonhq.helloandroid.MainActivity@364c0b9
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] V/GraalActivity(30675): loading substrate library
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] --------- beginning of crash
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] E/AndroidRuntime(30675): FATAL EXCEPTION: main
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] E/AndroidRuntime(30675): Process: com.soartech.isaacuserui, PID: 30675
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] E/AndroidRuntime(30675): java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "JNI_OnLoad_awt" referenced by "/data/app/com.soartech.isaacuserui-sc5Osp1K9aHqgnWO_iRKQw==/lib/arm64/libsubstrate.so"...
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] E/AndroidRuntime(30675):     at java.lang.Runtime.loadLibrary0(Runtime.java:1071)
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] E/AndroidRuntime(30675):     at java.lang.Runtime.loadLibrary0(Runtime.java:1007)
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] E/AndroidRuntime(30675):     at java.lang.System.loadLibrary(System.java:1667)
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] E/AndroidRuntime(30675):     at com.gluonhq.helloandroid.MainActivity.surfaceCreated(MainActivity.java:104)
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] E/AndroidRuntime(30675):     at android.view.SurfaceView.updateSurface(SurfaceView.java:926)
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] E/AndroidRuntime(30675):     at android.view.SurfaceView$2.onPreDraw(SurfaceView.java:215)
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] E/AndroidRuntime(30675):     at android.view.ViewTreeObserver.dispatchOnPreDraw(ViewTreeObserver.java:1102)
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] E/AndroidRuntime(30675):     at android.view.ViewRootImpl.performTraversals(ViewRootImpl.java:3310)
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] E/AndroidRuntime(30675):     at android.view.ViewRootImpl.doTraversal(ViewRootImpl.java:2200)
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] E/AndroidRuntime(30675):     at android.view.ViewRootImpl$TraversalRunnable.run(ViewRootImpl.java:9037)
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] E/AndroidRuntime(30675):     at android.view.Choreographer$CallbackRecord.run(Choreographer.java:999)
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] E/AndroidRuntime(30675):     at android.view.Choreographer.doCallbacks(Choreographer.java:797)
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] E/AndroidRuntime(30675):     at android.view.Choreographer.doFrame(Choreographer.java:732)
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] E/AndroidRuntime(30675):     at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:984)
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] E/AndroidRuntime(30675):     at android.os.Handler.handleCallback(Handler.java:883)
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] E/AndroidRuntime(30675):     at android.os.Handler.dispatchMessage(Handler.java:100)
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] E/AndroidRuntime(30675):     at android.os.Looper.loop(Looper.java:237)
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] E/AndroidRuntime(30675):     at android.app.ActivityThread.main(ActivityThread.java:8034)
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] E/AndroidRuntime(30675):     at java.lang.reflect.Method.invoke(Native Method)
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] E/AndroidRuntime(30675):     at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:496)
[Thu May 13 23:58:12 EDT 2021][INFO] [SUB] E/AndroidRuntime(30675):     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1076)

[oubidar-Abderrahim]

Does this issue occur with GraalVM CE java11 21.1.0 (without Gluon)?

[robert-picking]

After some digging, this appears to be an issue with using classes from the package java.beans which is in the java.desktop module which contains the package java.awt. I was able to confirm that removing the usage of these classes no longer results in the error above, which leads me to believe the inclusion of java.desktop module and by extension the java.awt package is the cause.

I believe you can reproduce this issue by including some usage of PropertyChangeListener interface or PropertyChangeSupport class from the package java.beans, and then attempting to build and run the app on android, which are the classes/interfaces that we were using.

[robert-picking]

Does this issue occur with GraalVM CE java11 21.1.0 (without Gluon)?

I've also confirmed that this issues still does occur with GraalVM CE java11 21.1.0 without gluon

[oubidar-Abderrahim]

Could you please provide a detailed step for reproducing the issue locally? thank you

[ctoabidmaqbool]

I am having exactly same issue too, Using

• WSL with Ubountu (windows 10)
• openjdk version "11.0.11" 2021-04-20
  OpenJDK Runtime Environment GraalVM 21.1.0-dev (build 11.0.11+5-jvmci-21.1-b03)
  OpenJDK 64-Bit Server VM GraalVM 21.1.0-dev (build 11.0.11+5-jvmci-21.1-b03, mixed mode, sharing)
• Android 11
• Client Maven Plugin: 0.1.41

[robert-picking]

I was able to reproduce this with only small changes to any project. For my example I was using the HelloFX example from gluon, but the cause of this seems to be adding the below reflection configuration that we required on our project. Without the reflection, this does not cause the java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "JNI_OnLoad_awt" referenced by... issue.

Adding the below apache-configuration2 dependency and including the below reflection config will result in the error specified above when the app is first launched on android.

Added pom dependency

<dependency>
    <groupId>org.apache.commons</groupId>
    <artifactId>commons-configuration2</artifactId>
    <version>2.7</version>
</dependency>

reflect-config.json

[
{
  "name":"org.apache.commons.configuration2.PropertiesConfiguration",
  "allPublicMethods":true,
  "allPublicConstructors":true
}
]

[oubidar-Abderrahim]

I'm having issues with reproducing since my Linux machine doesn't have much space for android-ndk.

I've also confirmed that this issues still does occur with GraalVM CE java11 21.1.0 without gluon

Could you please share how can I test it without Gluon?

[oubidar-Abderrahim]

reflect-config.json

[
{
  "name":"org.apache.commons.configuration2.PropertiesConfiguration",
  "allPublicMethods":true,
  "allPublicConstructors":true
}
]

When I use mvn client:build -Pandroid, it regenerates the config files and thus removes this entry, Could you please share the steps to build the native-image without losing the reflectionconfig.json content? Thank you

[yuehuaxingguang]

I am having exactly same issue too， details

[robert-picking]

reflect-config.json

[
{
  "name":"org.apache.commons.configuration2.PropertiesConfiguration",
  "allPublicMethods":true,
  "allPublicConstructors":true
}
]

When I use mvn client:build -Pandroid, it regenerates the config files and thus removes this entry, Could you please share the steps to build the native-image without losing the reflectionconfig.json content? Thank you

I've created a repo here that demonstrates this issue. You should be able to clone the repo, and run the command specified in the readme, which will compile, link, and package an apk, and then install and run the app on a device connected via adb. The app should install, but when opening it will immediately crash with the error described.

[oubidar-Abderrahim]

Tracked internally on GR-31945

[makingthematrix]

Hi,
I got this error when trying to run the following minimal example on Android: https://github.com/makingthematrix/scalaonandroid/tree/main/HelloFXGL
Even adding FXGL to pom.xml as a dependency, without using it in the code, is enough to make the app crash on init with java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "JNI_OnLoad_awt".

I talked to @AlmasB about it and it looks like the code works on GraalVM 20.2.0 - the error starts appearing from 21.1. On the other hand, 21.1 is required for the new GluonFX maven plugin.

[marinier]

I was getting this error, but using gluonfx maven plugin version 1.0.3 the error has gone away. The simple reproducer that @robert-picking posted is working now, too. I'm using gluon's build of graalvm (version 21.1, although there is a newer version now based on a pre-release of 21.2: https://github.com/gluonhq/graal/releases/tag/gluon-21.2.0-dev-20210621_1207).

[makingthematrix]

I was getting this error, but using gluonfx maven plugin version 1.0.3 the error has gone away. The simple reproducer that @robert-picking posted is working now, too. I'm using gluon's build of graalvm (version 21.1, although there is a newer version now based on a pre-release of 21.2: https://github.com/gluonhq/graal/releases/tag/gluon-21.2.0-dev-20210621_1207).

Thanks for the tip. I updated to gluonfx-maven-plugin 1.0.3, and GraalVM 21.2 (they have just realeased it) but I still get the same error. Same for gluon-21.2.0-dev. But I hope it means we're getting somewhere :)

[marinier]

You might try the simple example that @robert-picking posted: https://github.com/robert-picking/hellofx-jni-awt-bug

If that works, then it sounds like there's an additional bug somewhere. If it doesn't work, then it's possible there's a setup/configuration issue on your end. When I tested it, I was using Ubuntu 20.04.

[makingthematrix]

@marinier : I have just checked with that example. The bug still occurs. gluon-21.2.0-dev, gluonfx-maven-plugin 1.0.3. I'm on Linux Mint 20.1, but I don't think that's the issue here.

[marinier]

I just checked, and my java version is graalvm-svm-linux-gluon-21.1.0-dev, dated 4-20-2021.

I think the fix is actually in gluon's substrate, which should be part of the gluonfx plugin. (If you look at the commits to the gluonfx maven plugin, many of them are updating the substrate version.) I wonder if it's somehow picking up a different substrate version from the classpath? Like graalvm's?

[mipastgt]

I obviously just stumbled over the same error. Is there any fix or news in sight?

[marinier]

I just tried running @robert-picking's reproducer again and I'm now getting the awt error :( Not sure why it worked for me before. In my own application, there is a part that didn't work before that works now, but another part that still fails related to apache configuration2 (which is what the reproducer checks).

[mipastgt]

I am not using that apache library but still get this error.

[marinier]

It's not specific to that library. When we investigated before, it seemed like it might be related to code that uses classes from the java.desktop module (but not all classes). Apache configuration uses beans, which are in that module.

[mipastgt]

I am working on a pure JavaFX example right at the moment. I found that something like this should also cause the trouble but I still have to confirm that.

protected final DoubleBinding worldMetersPerPixel = Bindings.selectDouble(mc, "viewport", "worldMetersPerPixel");
com.sun.javafx.property.adapter.ReadOnlyPropertyDescriptor source: file:/Users/mpaus/.m2/repository/org/openjfx/javafx-base/17-ea+16/javafx-base-17-ea+16-mac.jar
java.beans.PropertyChangeListener source: jrt:/java.desktop

A simple select binding uses the ReadOnlyPropertyDescriptor which then uses the PropertyChangeListener which is in java.desktop.

UPDATE: I can't confirm this. In this context the problem does not show up although it uses java.desktop. So, just using java.desktop is not a sufficient condition to trigger the bug.

[marinier]

I think we ran into issues with PropertyChangeListener too. It seems sufficient to have an entry in the reflect-config.json file -- it doesn't seem to be necessary to actually use it in your code.

[jperedadnr]

There is a tentative workaround to avoid AWT references on Android, providing AWT is not used explicitly by the project.
To test it use:

gluonfx-maven-plugin version: 1.0.4-SNAPSHOT

plugin repository:

    <pluginRepositories>
        <pluginRepository>
            <id>Snapshots</id>
            <url>http://oss.sonatype.org/content/repositories/snapshots/</url>
        </pluginRepository>
    </pluginRepositories>

[marinier]

I tried it with @robert-picking's reproducer and it worked (for real this time).

[oubidar-Abderrahim]

Another workaround to keep in mind is using -H:DeadlockWatchdogInterval=60.
That being said, this issue is more related to Gluon. GraalVM does not support running native images on Android yet, and the Gluon team did a lot of work, and changes, to make this happen.