Multi-Crate project - second try

CHANGELOG.md

@@ -26,6 +26,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Allow creating a `QImage` from an `image::RgbaImage`.
 - Support for `cfg` attributes through to C++ generation
 - CXX-Qt-build: Improved compile time and propagation of initializers between crates
+- CXX-Qt-build: Multi-crate projects are now possible with Cargo and CMake (see `examples/qml_multi_crates`)
+- CXX-Qt-build: Allow forcing initialization of crates/QML modules (`cxx_qt::init_crate!`/`cxx_qt::init_qml_module!`)
 
 ### Fixed
 

Cargo.toml

@@ -18,6 +18,9 @@ members = [
     "examples/qml_features/rust",
     "examples/qml_minimal/rust",
     "examples/qml_basics",
+    "examples/qml_multi_crates/rust/main",
+    "examples/qml_multi_crates/rust/sub1",
+    "examples/qml_multi_crates/rust/sub2",
 
     "tests/basic_cxx_only/rust",
     "tests/basic_cxx_qt/rust",

book/src/SUMMARY.md

@@ -27,6 +27,7 @@ SPDX-License-Identifier: MIT OR Apache-2.0
   - [Shared types](./bridge/shared_types.md)
   - [Attributes](./bridge/attributes.md)
   - [Traits](./bridge/traits.md)
+- [Common Issues](./common-issues.md)
 - [For Contributors: CXX-Qt Internals](./internals/index.md)
   - [Build System](./internals/build-system.md)
   - [Crate Organization](./internals/crate-organization.md)

book/src/common-issues.md

@@ -0,0 +1,79 @@
+<!--
+SPDX-FileCopyrightText: 2025 Klarälvdalens Datakonsult AB, a KDAB Group company <[email protected]>
+SPDX-FileContributor: Leon Matthes <[email protected]>
+
+SPDX-License-Identifier: MIT OR Apache-2.0
+-->
+# Common Issues
+
+## Cargo Linker Error: Undefined reference to `cxx_qt_init_`
+
+CXX-Qt recreates Qt's resource initialization system within a mix of Cargo and CMake.
+
+This initialization system generates functions that are prefixed with `cxx_qt_init_crate` or `cxx_qt_init_qml_module`.
+
+When building with Cargo, under certain crate setups you may encounter errors that the linker cannot find these functions, e.g.:
+
+```shell
+= note: /.../out/cxx-qt-build/qml_module_com_kdab_cxx_qt_demo/call-initializers.cpp:2:
+          error: undefined reference to 'cxx_qt_init_qml_module_com_kdab_cxx_qt_demo'
+        /.../out/cxx-qt-build/initializers/crate_another_crate/call-initializers.cpp:2:
+          error: undefined reference to 'cxx_qt_init_crate_another_crate'
+        clang: error: linker command failed with exit code 1 (use -v to see invocation)
+
+  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
+  = note: use the `-l` flag to specify native libraries to link
+  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-link-lib)
+```
+
+To fix this issue, you need to make sure of two things:
+
+### 1. Ensure dependencies are used
+
+If a dependency is not used by the target currently being built, the Rust toolchain will not link to it.
+This is particularly common if a dependency provides a QML module creates types for use in QML that aren't actually needed by the Rust code of downstream crates.
+
+To fix this, force the Rust compiler to link to the crate by adding:
+
+```rust,ignore
+extern crate another_crate;
+```
+
+(where another_crate is replaced by the name of the dependency that isn't otherwise used).
+
+### 2. Include the initializers in your code
+
+Next we need to ensure the initializers can be found by the linker.
+
+If you followed step 1, modern linkers like `mold` or `lld` should already be able to link everything correctly.
+We encourage switching to such a linker if you're still using the (now deprecated) `ld.gold` on Linux.
+
+With older linkers, you can force initialization manually by calling the corresponding `init_` macros from the cxx_qt crate at startup.
+
+```rust,ignore
+fn main() {
+  cxx_qt::init_crate!(another_crate);
+  cxx_qt::init_qml_module!("com.kdab.cxx_qt.demo");
+}
+```
+
+Note that you will have to do the same in tests and doc-tests:
+
+````rust,ignore
+/// ```
+/// # cxx_qt::init_crate!(another_crate);
+/// # cxx_qt::init_qml_module!(another_crate);
+///
+/// X::do_something();
+/// ```
+struct X {}
+
+#[cfg(test)]
+mod tests {
+  #[test]
+  fn initialize_eependencies() {
+    cxx_qt::init_crate!(another_crate);
+    cxx_qt::init_qml_module!("com.kdab.cxx_qt.demo");
+  }
+}
+````

book/src/internals/build-system.md

@@ -19,46 +19,17 @@ Qt code often contains initialization code that is called by a static variable t
 
 However, when linking into a static library, and then linking into the main executable, the linker will discard everything from the library that isn't used by the main executable, including these static initializers, as they're never actually used and just exist to run their constructor code.
 
-There are multiple ways to solve this:
-
-- Export an object file and link that to the main binary. Object files are always included completely
-- Use the whole-archive linker flag which forces inclusion of every object within the static library.
-  - If we include the entire static lib generated by cargo, then we'll likely get duplicate symbols, as this really includes **everything** that your Rust code **may** need, even if you don't use it.
-  - This has caused some recent regressions with Rust 1.78+, where MSVC could no longer link CXX-Qt due to duplicate symbols
-  - The way to solve this is to only export the static initializers as a library and link that into CMake.
-- Manually calling the static initializer code
-  - This is basically what Q_INIT_RESOURCE and Q_IMPORT_PLUGIN do
-  - They call the registration method directly, which circumvents the static initializers and forces the static initializers to be linked if they would otherwise be discarded.
-
-At the moment we employ a mix of all methods.
-
-First and foremost, we wrap all our initializers into functions with well-defined names (starting with `cxx_qt_init`) and C-compatible signatures.
-This allows us to manually call the initializers from any point in the linker chain, which forces their inclusion.
-These initializer functions call the initializer functions from their upstream dependencies so that the entire dependency tree is initialized.
-
-However, we don't want to have to call the initializers manually in every resulting binary.
-To solve this, we use static initializers that simply call the initializer function of the crate/Qml module, thereby initializing all dependencies.
-As noted earlier, these static initializers are routinely optimized out by the linker.
-
-For Cargo builds we prevent this by linking all initializers with +whole-archive which forces all of them to be included.
-Experience has shown that this gives us the best compatibility overall, as linking object files to Cargo builds turned out to be quite finicky.
-As the initializers contain very few symbols themselves, this should also rarely lead to issues with duplicate symbols.
-
-In CMake we mirror Qts behavior, which is to build the static initializer as an `OBJECT` library.
-The initializer functions themselves are still built into the Rust static library and the `OBJECT` library must therefore link to it.
-This is taken care of by the `cxx_qt_import_crate`/`_import_qml_module` functions.
-
 ### Header files
 
 We want to make the generated headers available, not just to CMake, but also within dependents in the cargo build chain (e.g. your crate will probably want to depend on the headers produced by cxx-qt-lib).
 
 For this we need to export them to a stable directory so that both CMake and Cargo can find them.
 
-### (Optional) Integration with CMake
+# (Optional) Integration with CMake
 
 Somehow, all of this should be compatible with both CMake, and Cargo-only builds.
 
-## The plan (for now)
+# The plan (for now)
 
 After many rounds of refactoring this, we believe that we need to be able to share data between build scripts for this to work halfway ergonomically.
 
@@ -97,6 +68,37 @@ Next to the crates directory, there should be a `qml_modules` directory, which c
 
 Each module should include a `plugin_init.o`, `.qmltypes`, `qmldir`, and any other necessary files.
 
+## Initializers with Cargo and CMake
+
+There are multiple ways to solve the issues presented by static initializers:
+
+- Export an object file and link that to the main binary. Object files are always included completely.
+- Use the whole-archive linker flag which forces inclusion of every object within the static library.
+  - If we include the entire static lib generated by cargo, then we'll likely get duplicate symbols, as this really includes **everything** that your Rust code **may** need, even if you don't use it.
+  - This has caused some recent regressions with Rust 1.78+, where MSVC could no longer link CXX-Qt due to duplicate symbols
+  - The way to solve this is to only export the static initializers as a library and link that into CMake.
+- Manually calling the static initializer code
+  - This is basically what Q_INIT_RESOURCE and Q_IMPORT_PLUGIN do
+  - They call the registration method directly, which circumvents the static initializers and forces the static initializers to be linked if they would otherwise be discarded.
+
+At the moment we employ a mix of all methods.
+
+First and foremost, we wrap all our initializers into functions with well-defined names (starting with `cxx_qt_init`) and C-compatible signatures.
+This allows us to manually call the initializers from any point in the linker chain, which forces their inclusion.
+These initializer functions call the initializer functions from their upstream dependencies so that the entire dependency tree is initialized.
+
+However, we don't want to have to call the initializers manually in every resulting binary.
+To solve this, we use static initializers that simply call the initializer function of the crate/Qml module, thereby initializing all dependencies.
+As noted earlier, these static initializers are routinely optimized out by the linker.
+
+For Cargo builds we prevent this by linking all initializers with +whole-archive which forces all of them to be included.
+Experience has shown that this gives us the best compatibility overall, as linking object files to Cargo builds turned out to be quite finicky.
+As the initializers contain very few symbols themselves, this should also rarely lead to issues with duplicate symbols.
+
+In CMake we mirror Qts behavior, which is to build the static initializer as an `OBJECT` library.
+The initializer functions themselves are still built into the Rust static library and the `OBJECT` library must therefore link to it.
+This is taken care of by the `cxx_qt_import_crate`/`_import_qml_module` functions.
+
 ## Integration with CMake
 
 Via the `CXXQT_EXPORT_DIR` environment variable CMake should be able to change the location of the "target" directory.

crates/cxx-qt-build/src/dir.rs

@@ -40,17 +40,38 @@ pub(crate) fn crate_target() -> PathBuf {
     target().join("crates").join(crate_name())
 }
 
-/// The target directory, namespaced by plugin
+/// The target directory, namespaced by QML module
 pub(crate) fn module_target(module_uri: &str) -> PathBuf {
-    target()
-        .join("qml_modules")
-        .join(module_name_from_uri(module_uri))
+    module_export(module_uri).unwrap_or_else(|| {
+        out()
+            .join("qml_modules")
+            .join(module_name_from_uri(module_uri))
+    })
+}
+
+/// The export directory, namespaced by QML module
+///
+/// In conctrast to the crate_export directory, this is `Some` for downstream dependencies as well.
+/// This allows CMake to import QML modules from dependencies.
+///
+/// TODO: This may conflict if two dependencies are building QML modules with the same name!
+/// We should probably include a lockfile here to avoid this.
+pub(crate) fn module_export(module_uri: &str) -> Option<PathBuf> {
+    // In contrast to crate_export, we don't need to check for the specific crate here.
+    // QML modules should always be exported.
+    env::var("CXX_QT_EXPORT_DIR")
+        .ok()
+        .map(PathBuf::from)
+        .map(|dir| {
+            dir.join("qml_modules")
+                .join(module_name_from_uri(module_uri))
+        })
 }
 
 /// The target directory or another directory where we can write files that will be shared
 /// between crates.
 pub(crate) fn target() -> PathBuf {
-    if let Some(export) = export() {
+    if let Some(export) = crate_export() {
         return export;
     }
 
@@ -59,7 +80,7 @@ pub(crate) fn target() -> PathBuf {
 
 /// The export directory, if one was specified through the environment.
 /// Note that this is not namspaced by crate.
-pub(crate) fn export() -> Option<PathBuf> {
+pub(crate) fn crate_export() -> Option<PathBuf> {
     // Make sure to synchronize the naming of these variables with CMake!
     let export_flag = format!("CXX_QT_EXPORT_CRATE_{}", crate_name());
     // We only want to export this crate if it is the specific crate that CMake is looking for and
@@ -87,8 +108,8 @@ pub(crate) fn out() -> PathBuf {
     env::var("OUT_DIR").unwrap().into()
 }
 
-pub(crate) fn is_exporting() -> bool {
-    export().is_some()
+pub(crate) fn is_exporting_crate() -> bool {
+    crate_export().is_some()
 }
 
 pub(crate) fn initializers(key: &str) -> PathBuf {