[question] Please, am I missing something?

[KotlinGeekDev]

Hello everyone. I want to use a Rust library on Android. I am using Swig to generate JNI bindings and I am generating the shared library manually using a script(the clang toolchain). I am also manually copying the generated AAR into the app project. However, when running the app it shows the following error :

E/DemoApp: Error loading 'rgb' library: java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "_ZNSt12length_errorD1Ev" referenced by "/data/app/~~LN6GpR4dsQOH6xN0_cqfNA==/org.lnpbp.demoapp-1eB-I4WLe5-ims5tw7RtTA==/lib/x86/librgb.so"...
E/g.lnpbp.demoap: No implementation found for long org.lnpbp.rgb_autogen.rgbJNI.rgb_node_run(java.lang.String, java.lang.String, java.lang.String, short) (tried Java_org_lnpbp_rgb_1autogen_rgbJNI_rgb_1node_1run and Java_org_lnpbp_rgb_1autogen_rgbJNI_rgb_1node_1run__Ljava_lang_String_2Ljava_lang_String_2Ljava_lang_String_2S)

I have checked the AAR, SO files, and the symbol it complains about is present. Also, the function for which it can't find the implementation is also present. I have tried some fixes, all to no avail. I hope someone can help me.

[KotlinGeekDev]

For more context, this is the app project I want to run, and this is the bug I have run into, thus the error above. On a machine I am using(running Ubuntu 20.04), using an x86 android emulator running Android 9, I still run into the same error, but a different symbol not being found. Thanks for your time.

[DanAlbert]

demangler.com says that's std::length_error::~length_error(), which is not libc++. It looks like your library was built for a GNU system, not Android.

[DanAlbert]

Seems that the exception classes in stdexcept are actually not in a versioned namespace (which is why I assumed not libc++), so scratch that :)

[KotlinGeekDev]

OK. :). In the build_rust.sh script, libc++_shared.so is copied in order to be used. Please, do you suggest trying out libc++?

[DanAlbert]

In the bug you linked I see:

I.e. the reason of the error is that libc++_shared is not part of the APK (while present in the aar produced from bindings). This is the problem of the demo app; unfortunately I have not found how to fix it yet.

I'm not sure what build_rust.sh is (I haven't had time to read through the project you linked, just the bug). If you can upload the broken APK it'll probably be pretty obvious what's wrong, given the above comment.

[KotlinGeekDev]

Actually, I double-checked it, and now the .so is part of the apk, but still running into the original issue.

[KotlinGeekDev]

By the way, this is the script: here

[DanAlbert]

Can you upload the APK?

[KotlinGeekDev]

OK.

[KotlinGeekDev]

I can't upload the apk due to GitHub file type support. Sorry.

[DanAlbert]

Rename it .txt

[KotlinGeekDev]

app-debug.txt

[DanAlbert]

$ readelf -dW lib/x86/librgb.so

Dynamic section at offset 0xc4e7dc contains 30 entries:
  Tag        Type                         Name/Value
 0x00000003 (PLTGOT)                     0xc4fd1c
 0x00000002 (PLTRELSZ)                   1456 (bytes)
 0x00000017 (JMPREL)                     0x2d55c
 0x00000014 (PLTREL)                     REL
 0x00000011 (REL)                        0x2b4c
 0x00000012 (RELSZ)                      174608 (bytes)
 0x00000013 (RELENT)                     8 (bytes)
 0x6ffffffa (RELCOUNT)                   21675
 0x00000006 (SYMTAB)                     0x1cc
 0x0000000b (SYMENT)                     16 (bytes)
 0x00000005 (STRTAB)                     0x151c
 0x0000000a (STRSZ)                      4704 (bytes)
 0x6ffffef5 (GNU_HASH)                   0x277c
 0x00000001 (NEEDED)                     Shared library: [liblog.so]
 0x00000001 (NEEDED)                     Shared library: [libstdc++.so]
 0x00000001 (NEEDED)                     Shared library: [libdl.so]
 0x00000001 (NEEDED)                     Shared library: [libc.so]
 0x00000001 (NEEDED)                     Shared library: [libm.so]
 0x0000001a (FINI_ARRAY)                 0xc1a100
 0x0000001c (FINI_ARRAYSZ)               8 (bytes)
 0x00000019 (INIT_ARRAY)                 0xc4f7d8
 0x0000001b (INIT_ARRAYSZ)               4 (bytes)
 0x0000001e (FLAGS)                      BIND_NOW
 0x6ffffffb (FLAGS_1)                    Flags: NOW
 0x6ffffff0 (VERSYM)                     0x2824
 0x6ffffffc (VERDEF)                     0x2a90
 0x6ffffffd (VERDEFNUM)                  1
 0x6ffffffe (VERNEED)                    0x2aac
 0x6fffffff (VERNEEDNUM)                 4
 0x00000000 (NULL)                       0x0

You included libc++_shared.so but you didn't actually link against it.

$ readelf -sW lib/x86/librgb.so | c++filt | grep std
    47: 00000000     0 OBJECT  GLOBAL DEFAULT  UND stdin@LIBC (3)
   189: 00000000     0 OBJECT  GLOBAL DEFAULT  UND stderr@LIBC (3)
   192: 00000000     0 OBJECT  GLOBAL DEFAULT  UND std::nothrow@LIBC_O (7)
   194: 00000000     0 FUNC    GLOBAL DEFAULT  UND operator new(unsigned int, std::nothrow_t const&)@LIBC_O (7)
   195: 00000000     0 FUNC    GLOBAL DEFAULT  UND operator delete(void*, std::nothrow_t const&)@LIBC_O (7)
   248: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::logic_error::logic_error(char const*)
   249: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::length_error::~length_error()
   250: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND typeinfo for std::length_error
   251: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND vtable for std::length_error
   256: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::terminate()
   257: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND typeinfo for std::bad_alloc
   260: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::__ndk1::locale::use_facet(std::__ndk1::locale::id&) const
   261: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::__ndk1::ios_base::getloc() const
   262: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::__ndk1::ctype<char>::id
   263: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::__ndk1::locale::locale()
   264: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::__ndk1::locale::~locale()
   265: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::__ndk1::num_put<char, std::__ndk1::ostreambuf_iterator<char, std::__ndk1::char_traits<char> > >::id
   266: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::__ndk1::ios_base::__set_badbit_and_consider_rethrow()
   267: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::__ndk1::ios_base::init(void*)
   268: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::__ndk1::ios_base::clear(unsigned int)
   269: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::__ndk1::ios_base::~ios_base()
   270: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::uncaught_exception()
   271: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND typeinfo for std::__ndk1::ios_base
   275: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::out_of_range::~out_of_range()
   276: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND typeinfo for std::out_of_range
   277: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND vtable for std::out_of_range
   278: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::__ndk1::__shared_weak_count::__get_deleter(std::type_info const&) const
   279: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::__ndk1::condition_variable::notify_all()
   280: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::__ndk1::condition_variable::__do_timed_wait(std::__ndk1::unique_lock<std::__ndk1::mutex>&, std::__ndk1::chrono::time_point<std::__ndk1::chrono::system_clock, std::__ndk1::chrono::duration<long long, std::__ndk1::ratio<1ll, 1000000000ll> > >)
   281: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::__ndk1::condition_variable::wait(std::__ndk1::unique_lock<std::__ndk1::mutex>&)
   282: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::__ndk1::condition_variable::~condition_variable()
   283: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::__ndk1::__shared_weak_count::__release_weak()
   284: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::__ndk1::__shared_weak_count::~__shared_weak_count()
   285: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::__ndk1::__throw_system_error(int, char const*)
   286: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::__ndk1::mutex::lock()
   287: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::__ndk1::mutex::unlock()
   288: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::__ndk1::mutex::~mutex()
   289: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::__ndk1::chrono::steady_clock::now()
   290: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND std::__ndk1::chrono::system_clock::now()
   291: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND typeinfo for std::__ndk1::__shared_weak_count

But you're definitely using it. Whatever build system built librgb.so (cargo?) isn't using -Wl,--no-undefined, so your build failures are being deferred to runtime.

You're either missing a flag when building that library (and the build system isn't protecting you from that like it ought to) or the build system that built it is just wrong. The peculiar part is that it's pretty clearly using the headers from libc++, it's just never linking it. Maybe the libc++ dependency is coming in from a static library being linked into librgb.so, and librgb.so doesn't know about that?

[KotlinGeekDev]

I am manually building it using clang++ directly.

[DanAlbert]

Are you using an old NDK?

[KotlinGeekDev]

By the way, I am copying libc++_shared from Sdk/ndk/20.1.5948944/sources/cxx-stl/llvm-libc++/libs/x86/

[KotlinGeekDev]

NDK v. 20.1.5948944

[DanAlbert]

I am manually building it using clang++ directly.

And then immediately clobbering it with something built by cargo: https://github.com/rgb-org/rgb-sdk/blob/a61df14e1c4e3eab739cffef4034187e756d89ce/bindings/android/build_rust.sh#L28-L29

[KotlinGeekDev]

Please, I don't understand. Please, do you suggest I remove the second line(the line that copies the lib)?

[DanAlbert]

That's at least one of your problems. The problem with cargo is another, and I haven't figured that one out yet.

[DanAlbert]

Whether you need to the first line, the second line, or both and you need to rename one of the libraries, I have no idea. I'm not sure what these libraries are, all I know is that you're building one correctly and then overwriting it with one that's built incorrectly.

[KotlinGeekDev]

OK, I see.

[KotlinGeekDev]

I will have to test the two cases to be sure, then.

[KotlinGeekDev]

Please, can you go into more detail about the problem with cargo?(I hope I am not asking too much :) )

[DanAlbert]

I will once I understand it. It's building the broken library, that's all I've figured out so far.

The flags you're setting when calling cargo are certainly wrong: https://github.com/rgb-org/rgb-sdk/blob/a61df14e1c4e3eab739cffef4034187e756d89ce/bindings/android/build_rust.sh#L16. See https://developer.android.com/ndk/guides/other_build_systems. You've passed -nostdlib++ but are also configuring the libc++ headers. Your library has references to libc++ but is not linked against it because that's what you've told clang to do.

But if I correct them (by deleting them) CMake (idk how or why cmake fits into this) fails to configure the target correctly, and also can't find the linker. Haven't figured that one out yet.

You've also mixed and matched toolchains: https://github.com/rgb-org/rgb-sdk/blob/a61df14e1c4e3eab739cffef4034187e756d89ce/bindings/android/build_rust.sh#L11 is using API 26, but elsewhere you've used API 21. idk if that's a part of the problem but there's definitely no reason to do that.

Where is any of the cargo for android behavior documented? I had to rustup target add aarch64-linux-android to get your script running at all but I can't figure out where that target is actually implemented.

[DanAlbert]

tl;dr your dependencies haven't been ported to Android.

This isn't a valid Android CMake build. It doesn't follow https://developer.android.com/ndk/guides/cmake or https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html#cross-compiling-for-android. No -DCMAKE_TOOLCHAIN_FILE, no -DCMAKE_SYSTEM_NAME

"cmake" "/usr/local/google/home/danalbert/.cargo/git/checkouts/zeromq-src-rs-da612f523629c522/e95a4e6/vendor" "-DCMAKE_INSTALL_LIBDIR=lib" "-DCMAKE_C_STANDARD=99" "-DZMQ_BUILD_TESTS=OFF" "-DENABLE_DRAFTS=OFF" "-DENABLE_CURVE=ON" "-DCMAKE_BUILD_TYPE=Release" "-DWITH_PERF_TOOL=OFF" "-DBUILD_SHARED=OFF" "-DBUILD_STATIC=ON" "-DWITH_LIBSODIUM=OFF" "-DCMAKE_INSTALL_PREFIX=/work/src/rgb-sdk/librgb/target/aarch64-linux-android/release/build/zmq-sys-12267c93c85bdab5/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC --target=aarch64-linux-android" "-DCMAKE_C_COMPILER=/usr/local/google/home/danalbert/src/android-ndk-r22b/toolchains/llvm/prebuilt/linux-x86_64/bin/clang" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC --target=aarch64-linux-android" "-DCMAKE_CXX_COMPILER=/usr/local/google/home/danalbert/src/android-ndk-r22b/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC --target=aarch64-linux-android" "-DCMAKE_ASM_COMPILER=/usr/local/google/home/danalbert/src/android-ndk-r22b/toolchains/llvm/prebuilt/linux-x86_64/bin/clang"

That happens when building zeromq, on of your dependencies. https://github.com/LNP-BP/zeromq-src-rs/blob/fix/cmake/src/lib.rs seems to be what's calling CMake, and it's wrong for Android. The CMake crate is correct (https://github.com/alexcrichton/cmake-rs/blob/c87b73114d3d4b496018c0c5b8200d0274821a6e/src/lib.rs#L383-L391), but the zeromq-rs crate is not setting it up properly. The zeromq build doesn't appear to be configurable: https://github.com/LNP-BP/zeromq-src-rs/blob/e95a4e68136a6897d61c936622933bf259ebdd7f/src/lib.rs#L215-L335

[DanAlbert]

btw, a reduced (I removed most ABIs for debugging) version of your build script that fixes the obvious problems:

#!/usr/bin/env bash
set -eo pipefail
set -x

RUSTLIB="../../librgb"

cargo build --manifest-path $RUSTLIB/Cargo.toml

TOOLCHAIN=$NDK_HOME/toolchains/llvm/prebuilt/linux-x86_64/bin

export CARGO_TARGET_AARCH64_LINUX_ANDROID_LINKER="$TOOLCHAIN/clang++ -target aarch64-linux-android21"

AR="$TOOLCHAIN/llvm-ar" CC="$TOOLCHAIN/clang -target aarch64-linux-android21" CXX="$TOOLCHAIN/clang++ -target aarch64-linux-android21" cargo build --manifest-path $RUSTLIB/Cargo.toml --target=aarch64-linux-android --release

JNILIBS="library/src/main/jniLibs"

mkdir -pv library/src/main/java/org/lnpbp/rgb_autogen

mkdir -p $JNILIBS/arm64-v8a $JNILIBS/x86_64 $JNILIBS/armeabi-v7a $JNILIBS/x86

cp -v $RUSTLIB/target/aarch64-linux-android/release/librgb.so $JNILIBS/arm64-v8a/

[KotlinGeekDev]

Thanks a lot 🙏

[KotlinGeekDev]

Please, what about Swig, since it is used to generate the JNI bindings?(Because I dont see it in the reduced version)

[DanAlbert]

That part was working fine, you were just clobbering the output. Don't do that and it was building fine.

[KotlinGeekDev]

OK. Thanks very much. I will try these out more extensively, and get back to you 🙏 👍

[KotlinGeekDev]

Hello again. I tried and this time, it still crashed with the following error:

E/DemoApp: Error loading 'rgb' library: java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "_ZNSt6__ndk15ctypeIcE2idE" referenced by "/data/data/org.lnpbp.demoapp/app_lib/librgb.so"...
E/g.lnpbp.demoap: No implementation found for long org.lnpbp.rgb_autogen.rgbJNI.rgb_node_run(java.lang.String, java.lang.String, java.lang.String, short) (tried Java_org_lnpbp_rgb_1autogen_rgbJNI_rgb_1node_1run and Java_org_lnpbp_rgb_1autogen_rgbJNI_rgb_1node_1run__Ljava_lang_String_2Ljava_lang_String_2Ljava_lang_String_2S)

[KotlinGeekDev]

Right now, I want to add a CMake toolchain file. Please, @DanAlbert , should I add it to the library project or to the app project?

[DanAlbert]

You don't add it anywhere, you use the one from the NDK. You'd need to fix the zeromq crate I linked before.

[KotlinGeekDev]

Please, do you have any pointers on how to fix it?(Since I am not very experienced)

[DanAlbert]

#1508 (comment) points to the broken sources, broken functions within those sources, and the docs that explain how to use cmake correctly.

[KotlinGeekDev]

OK, thanks very much. Did not pay close attention to the comment.

[KotlinGeekDev]

So, I will have to basically fork the zeromq crate, apply the corrections, and test it out.

[DanAlbert]

Actually, it looks like the CMake crate will accept a toolchain file from the environment: https://github.com/alexcrichton/cmake-rs/blob/c87b73114d3d4b496018c0c5b8200d0274821a6e/src/lib.rs#L727-L731

[KotlinGeekDev]

So does it mean that I just need to create a toolchain file and point to it in the script?

[DanAlbert]

It looks like it's because the project is stuck on an ancient version of the cc crate that doesn't work with the NDK. You need to update those dependencies. Not really anything to do with the NDK, so you'll need to take those bugs to the project owner.

[KotlinGeekDev]

Please, which project? zero-mq?

[DanAlbert]

cargo tree will show you the dependencies of the project.

[KotlinGeekDev]

Thanks, I saw. I'll report it then. :)

[KotlinGeekDev]

I think I should not also forget to pass in the swig C++ generated file as an argument to clang in the script.