Thoughts on no-lcm support

[gareth-cross]

@aaron-skydio @hayk-skydio Wasn't sure whether this ought to be a "discussion" or an "issue", but I opted for the former since this felt less concrete for the time being.

At present, I would like to use SymForce w/o necessarily using LCM (or skymarshal, more accurately). I am curious if you have any interest in supporting this (or merging changes that support for it). It seems somewhat likely that other eventual users might share this ambition. Some rationale:

• I would prefer to code-generate C++ and Rust structs directly w/ a custom syntax.
• Another use case might be code-generating some other intermediate format (ROS messages, maybe).
• In general, it moves the framework more towards a world where users can create a "plugin" for their ecosystem (custom language + custom message format), while still retaining the core framework.

There are certainly some constraints here - for example the generated structs still have to compile with the generated code. That said, the SymForce generated code imposes relatively few demands on the structures. In C++, for example, any POD type that supports foo.bar.thing[2] = ...; syntax should work. The rest of the generated LCM boilerplate is somewhat orthogonal to SymForce, at least for external users.

Much like the target language can be customized via CodegenConfig after this merges, it would be convenient if the message format could also be customized.

I have attempted to make a minimal change here that does the following:

• Adds generate_function_no_lcm which skips steps involving LCM.
• Pulled out some shared logic from generate_types so it is re-used in generate_function_no_lcm and generate_function.
• Define TypeDescription, a formalization of the types_dict object for the sake of legibility.
• generate_function_no_lcm returns the types_dict so some user code can then deal with generation of types.

This change is just a simple way of starting to attack the issue, which only aims to give the user the option of taking responsibility for type generation while minimally impacting the existing workflow. To really make this work for a broader pool of users, some other changes might be required. For example, adding some ability to customize include paths in existing languages like C++/Python. Right now they are hardcoded to <lcmtypes/...>, and the only vehicle for changing this is editing the jinja templates. It would be convenient if instead the user could inherit from CppCodegenConfig and override a method that resolves include paths (one hypothetical alternative).

In principle, one could introduce a TypeCodegenConfig class that encapsulates type-generation behavior - much like CodegenConfig would eventually do for different languages, at least after Hayk's change merges. The current LCM behavior would be a specific sub-class, and third party users could provide yet more implementations for their ecosystem. I imagine ROS might be a popular choice for researchers (just a guess). Interested in getting your thoughts on this.

[hayk-skydio]

Short answer here is that we are interested in this sort of plugin system and have discussed it in the past, since LCM is not widespread. I believe nothing in the core depends on LCM-specific features that a POD struct wouldn't have.

For serialization purposes ROS or protobuf probably make the most sense, and I also understand wanting plain structs that don't serialize but the types can be customized.

I haven't looked at your change yet - my main desire is to minimally complicate the API. Thanks for giving it a go. Also, the codegen centralization change (at least the first pass) is merged.