Initial Android MLS bindings (#356)

Have extremely basic binding working in the Android example app (`create_client`), just putting this up first. Also got persistent DB storage working. Not quite ready to integrate yet - there's still more pieces to do here:

1. Get DB encryption to work (seems to be broken in general, not just with the bindings) - this PR disables encryption for now
2. Expose more API surface (and unit tests)
3. Android integration tests
4. Tidy up API surface and naming

bindings_ffi/Cargo.toml

@@ -16,7 +16,7 @@ uniffi = { git = "https://github.com/mozilla/uniffi-rs", rev = "cae8edc45ba5b56b
   "cli",
 ] }
 uniffi_macros = { git = "https://github.com/mozilla/uniffi-rs", rev = "cae8edc45ba5b56bfcbf35b60c1ab6a97d1bf9da" }
-xmtp = { path = "../xmtp", features = ["grpc", "native"] }
+xmtp_mls = { path = "../xmtp_mls", features = ["grpc", "native"] }
 xmtp_cryptography = { path = "../xmtp_cryptography" }
 xmtp_api_grpc = { path = "../xmtp_api_grpc" }
 xmtp_v2 = { path = "../xmtp_v2" }

bindings_ffi/README.md

@@ -6,14 +6,16 @@ This crate provides cross-platform Uniffi bindings for XMTP v3.
 
 - Android is tested end-to-end via an example app in `../examples/android`.
 - iOS has not been tested.
+- Dev network is NOT supported, only local network is currently supported.
 
-# Consuming this crate
+# Consuming this crate (Android example)
 
 The generated artifacts of this crate are the bindings interface (`xmtpv3.kt`) generated by `uniffi`, and the cross-compiled binaries (`jniLibs/`) generated by `cross`. The binaries are checked into source so that you don't have to recompile.
 
 - Run `./gen_kotlin.sh` to generate the bindings interface.
 - Run `./setup_android_example.sh` to copy these artifacts into the example Android app. Alternatively, modify the script to set up an app of your choice.
 - Open the `build.gradle` of the example Android app in Android Studio.
+- Run the local server via `dev/up` from the root of this repo. If running from elsewhere, make sure your `docker-compose.yml` matches the one in this repo.
 
 # Rebuilding this crate
 
@@ -38,19 +40,36 @@ You'll need to do the following one-time setup to run Kotlin tests:
 
 If you want to skip the setup, you can also run `cargo test -- --skip kts` to only run Rust unit tests. CI will run all tests regardless.
 
+# Debugging
+
+There is no support for using breakpoints or a debugger with FFI currently. Methods of debugging:
+
+1. Examine any error messages that are returned from libxmtp.
+1. Produce a minimal repro of the issue.
+1. Use platform-native logging:
+   1. Set up an FFI logger for your platform ([example](https://github.com/xmtp/libxmtp/blob/7e7bf7aabe7c758507ae982834d583c1d88c3ce2/bindings_ffi/examples/MainActivity.kt#L33))
+   1. Add logs where you need them ([example](https://github.com/xmtp/libxmtp/assets/696206/bb1be87e-7a9b-47f2-a0f4-e93a92346b18))
+   1. Examine the logs for your platform (for example, on Android emulator this would be logcat in Android Studio)
+1. Examine the database in your app
+   1. Use logs to find the location of the Sqlite database and the database encryption key
+   1. Find the database (for example, on Android emulator use Device File Explorer in Android Studio)
+   1. Copy the database to your local machine and open it on the command line using `sqlite3`
+   1. Decrypt the database if needed [as follows](https://utelle.github.io/SQLite3MultipleCiphers/docs/configuration/config_sql_pragmas/#pragma-key) (or disable database encryption before running the app)
+
 # Releasing new version
 
 Tag the commit you want to release with the appropriate version (e.g. 0.3.0-beta0).
 The Release github workflow will run the following jobs:
 
 - android
-    - downloads the `libxmtp-android.zip` build artifact
-    - make a release tagged the same way with the artifact attached
+
+  - downloads the `libxmtp-android.zip` build artifact
+  - make a release tagged the same way with the artifact attached
 
 - swift
-    - downloads the `libxmtp-swift.zip`` build artifact
-    - checks out `libxmtp-swift` repo and updates it with the contents of the zip file
-    - pushes new commit to the `libxmtp-swift` repo and tags it with the same tag
+  - downloads the `libxmtp-swift.zip`` build artifact
+  - checks out `libxmtp-swift` repo and updates it with the contents of the zip file
+  - pushes new commit to the `libxmtp-swift` repo and tags it with the same tag
 
 NOTES: To allow the workflow to push to another repo the setup follows [this guide](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/making-authenticated-api-requests-with-a-github-app-in-a-github-actions-workflow#authenticating-with-a-github-app). It uses [this app installed on the org](https://github.com/organizations/xmtp/settings/apps/libxmtp-release). The relevant secrets are stored only [in this repo](https://github.com/xmtp/libxmtp/settings/secrets/actions). If additional repos are added to this workflow they MUST be added to [this installation](https://github.com/organizations/xmtp/settings/installations/39118494) of the app.
 
@@ -66,6 +85,6 @@ Any objects crossing the Uniffi interface boundary must be wrapped in `Arc<>`, s
 
 ## Async and concurrency
 
-We use Tokio as our [async runtime](https://rust-lang.github.io/async-book/08_ecosystem/00_chapter.html). Uniffi can use this runtime on async methods and objects using the annotation `#[uniffi::export(async_runtime = ‘tokio’)]`. Uniffi plumbs up an executor (scheduler) in the foreign language to the Tokio runtime in Rust. More details [here](https://github.com/mozilla/uniffi-rs/blob/734050dbf1493ca92963f29bd3df49bb92bf7fb2/uniffi_core/src/ffi/rustfuture.rs#L11-L18).
+We use Tokio as our [async runtime](https://rust-lang.github.io/async-book/08_ecosystem/00_chapter.html). Uniffi can use this runtime on async methods and objects using the annotation `#[uniffi::export(async_runtime = ‘tokio’)]`. Uniffi plumbs up an executor (scheduler) in the foreign language to the Tokio runtime in Rust. More details [here](https://github.com/mozilla/uniffi-rs/blob/734050dbf1493ca92963f29bd3df49bb92bf7fb2/uniffi_core/src/ffi/rustfuture.rs#L11-L18). By default, Tokio's [multi-thread scheduler](https://docs.rs/tokio/latest/tokio/runtime/index.html#multi-thread-scheduler) is used on the Rust side. This means that functions may resume execution in a different thread when an `await` is encountered. This in turn means that structs held across an `await` in libxmtp must be `Send` and `Sync`.
 
-Because the foreign language may be multi-threaded, any objects passed to the foreign language must be `Send` and `Sync`, and [no references to `&mut self` are permitted](https://mozilla.github.io/uniffi-rs/udl/interfaces.html#concurrent-access). For now, use the mutability pattern described in https://github.com/xmtp/libxmtp/pull/138.
+Additionally, because the foreign language may be multi-threaded, any objects passed to the foreign language must also be `Send` and `Sync`, and [no references to `&mut self` are permitted](https://mozilla.github.io/uniffi-rs/udl/interfaces.html#concurrent-access). For now, use the mutability pattern described in https://github.com/xmtp/libxmtp/pull/138.

bindings_ffi/examples/MainActivity.kt

@@ -11,10 +11,11 @@ import org.web3j.crypto.ECKeyPair
 import org.web3j.crypto.Sign
 import uniffi.xmtpv3.FfiInboxOwner
 import uniffi.xmtpv3.FfiLogger
+import java.io.File
 import java.nio.charset.StandardCharsets
 import java.security.SecureRandom
 
-const val EMULATOR_LOCALHOST_ADDRESS = "http://10.0.2.2:5556"
+const val DEV_NETWORK_ADDRESS = "https://dev.xmtp.network:5556"
 
 class Web3jInboxOwner(private val credentials: Credentials) : FfiInboxOwner {
     override fun getAddress(): String {
@@ -45,14 +46,29 @@ class MainActivity : AppCompatActivity() {
         val privateKey: ByteArray = SecureRandom().generateSeed(32)
         val credentials: Credentials = Credentials.create(ECKeyPair.create(privateKey))
         val inboxOwner = Web3jInboxOwner(credentials)
+        val dbPath: String = this.filesDir.absolutePath + "/android_example.db3"
+        val dbEncryptionKey: List<UByte> = SecureRandom().generateSeed(32).asUByteArray().asList()
+        Log.i(
+            "App",
+            "INFO -\nprivateKey: " + privateKey.asList() + "\nDB path: " + dbPath + "\nDB encryption key: " + dbEncryptionKey
+        )
 
         runBlocking {
             try {
-                val client = uniffi.xmtpv3.createClient(AndroidFfiLogger(), inboxOwner, EMULATOR_LOCALHOST_ADDRESS, false);
-                textView.text = "Client constructed, wallet address: " + client.walletAddress();
+                val client = uniffi.xmtpv3.createClient(
+                    AndroidFfiLogger(),
+                    inboxOwner,
+                    DEV_NETWORK_ADDRESS,
+                    true,
+                    dbPath,
+                    dbEncryptionKey
+                )
+                textView.text = "Client constructed, wallet address: " + client.accountAddress()
             } catch (e: Exception) {
-                textView.text = "Failed to construct client: " + e.message;
+                textView.text = "Failed to construct client: " + e.message
             }
         }
+
+        File(dbPath).delete()
     }
-}
+}

bindings_ffi/src/inbox_owner.rs

@@ -27,7 +27,7 @@ impl RustInboxOwner {
     }
 }
 
-impl xmtp::InboxOwner for RustInboxOwner {
+impl xmtp_mls::InboxOwner for RustInboxOwner {
     fn get_address(&self) -> String {
         self.ffi_inbox_owner.get_address().to_lowercase()
     }