[Rust] generated code doesn't support multiple schema files

[rnarubin]

Generated Rust code encounters compiler errors when referencing types defined in different schema files. Consider this example:

first_file.fbs:

namespace first_namespace;
table FirstFileTable {}

second_file.fbs:

include "first_file.fbs";
namespace second_namespace;
table SecondFileTable {
  firstTableInSecondTable: first_namespace.FirstFileTable;
}

main.rs:

mod second_file_generated;
fn main() {}

$ flatc --rust first_file.fbs second_file.fbs && cargo build:

error[E0433]: failed to resolve: could not find `first_namespace` in `super`
  --> src/second_file_generated.rs:56:58
   |
56 |   pub fn firstTableInSecondTable(&self) -> Option<super::first_namespace::FirstFileTable<'a>> {
   |                                                          ^^^^^^^^^^^^^^^ could not find `first_namespace` in `super`

flatc version 1.10.0
cargo 1.33.0 (f099fe94b 2019-02-12)

I think the fix for this issue is simple, the generated sources need additional mod declarations for the different files they access.

What's more complicated, however, is when the same namespace is used in these separate schemas:

first_file.fbs:

namespace shared_namespace;
table FirstFileTable {}

second_file.fbs:

include "first_file.fbs";
namespace shared_namespace;
table SecondFileTable {
  firstTableInSecondTable: shared_namespace.FirstFileTable;
}

in which case the types aren't qualified in the generated source:

error[E0412]: cannot find type `FirstFileTable` in this scope
  --> src/shared_second_file_generated.rs:56:51
   |
56 |   pub fn firstTableInSecondTable(&self) -> Option<FirstFileTable<'a>> {
   |                                                   ^^^^^^^^^^^^^^ not found in this scope

Unlike e.g. C++ namespaces or Java packages, Rust modules of the same name in different files are distinct, so these usages would probably need to have qualified types. Then it becomes similar to the first problem of cross-file type access.

[rw]

@rnarubin Thanks for surfacing this! We've known about the lack of this feature since the Rust port was merged. You can see here that I added a TODO for myself last year to add this feature:

    // TODO(rw): make generated sub-namespace files importable

https://github.com/google/flatbuffers/blob/master/tests/rust_usage_test/tests/integration_test.rs#L1577

Would you have any interest in trying to get this working with a PR?

[rnarubin]

I see. I'd consider trying to fix this if it's unaddressed for a while longer, but in the near term we've settled on dumb workaround of stripping includes and concating all the files in build.rs

[nevi-me]

Is it a purely rust thing to get it working, or does one have to also write some cpp? It's something I'd also like to work on, maybe with a bit of guidance

[rnarubin]

This requires changes to the code generator, so it's principally cpp work.
I think the right approach is to structure modules as directories, where identical packages in different schemas will map to the same module directory. Then table/enums/structs could be defined in their own .rs files and declared in the module with a shared mod.rs file of the directory.
The existing code generator is geared around producing independent output files from scratch, however, so this kind of fix involving shared output files would require some refactoring or special cases that could quickly become complicated.

[stale]

This issue has been automatically marked as stale because it has not had activity for 1 year. It will be automatically closed if no further activity occurs. To keep it open, simply post a new comment. Maintainers will re-open on new activity. Thank you for your contributions.

[mwu-tow]

Still broken as of 1.12.0.

[wdanilo]

Hi! This is really crucial for us and this is currently a serious limitation. Do you have any estimate when it could be done? We are just moving some of our code to Rust and this causes serious problems on our side right now.

[moiseevigor]

Subscribe to this. Are there any updates to this issue?

[github-actions]

This issue is stale because it has been open 6 months with no activity. Please comment or this will be closed in 14 days.

[plwalsh]

Any plans to support this in the future?

[CasperN]

I definitely want to, but I can't promise a timeline :/

The current high level plan for this issue is: #5589 (comment)

Note that this interacts with some other efforts like build.rs integration, #5216, and a refactoring of flatc to use an IR #6428, so there may be some throwaway work / churn, but that's probably worth it.

[CasperN]

I think this is fixed at HEAD by #6731