Lua Generator using IR.

[dbaileychess]

Address: #6926

This implements the first generator using the the reflection.fbs schema as an IR between the flatc parser and code gen. It should serve as a model of how to write other generators using this approach.

I tried to keep things as well separated as possible, where this new generator only relies on the public flatbuffer.h headers and generated reflection_generated.h and knows nothing about the internals of the flatc parser. This required modifying the reflection.fbs a bit to provide some missing information. The tests/monster_test_bfbs_generated.h shows only a minor size increase.

The main interface for generation is the BfbsGenerator interface that has a single method:

GeneratorStatus generate(const uint8_t *buffer, int64_t length) = 0;

This simulates some random process reading in the IR buffer from somewhere (e.g., file, disk, memory). To avoid needing a separate binary, I integrated this new generator directly in the flatc binary, and pass the IR as a in-mem struct.

I kept the API as simple as possible, so that the generators can be implemented in whatever fashion they want, as long as they use this BfbsGenerator interface. For this Lua generator, I did make a BaseBfbsGenerator class that fulfills the BfbsGenerator interface and has some utility methods that should be applicable to any language generator.

This generator replaces the one for --lua, and I'll remove the old idl based on in a follow up CL.

I did modify the output of the generated Lua files a bit, so that is why there is a large diff between the two. But ignoring white space and some name simplifications, the core logic remains the identical.

I ran tests/LuaTest.sh with all the supported Lua flavors and they all passed:

[luajit]
# of test passed: 7 / 7 (100.00%)
[lua5.1]
# of test passed: 7 / 7 (100.00%)
[lua5.2]
# of test passed: 7 / 7 (100.00%)
[lua5.3]
# of test passed: 7 / 7 (100.00%)
[lua5.4]
# of test passed: 7 / 7 (100.00%)

[aardappel]

some quick initial thoughts

[aardappel]

This is going to be very much a preference thing, but this feels like a very verbose style of codegen to me. That would be ok if it was just for Lua, but by putting these functions in the base class you're implying that all such generators should use this style.

What should we use instead? Frankly, if its just down to me, I would not have such functions at all, and just += raw strings, including embedded indentation spaces. I think all this procedural indenting is overabstraction for any language that has a clear code style. Even for C++ we just output Google Style code, so it doesn't matter. And not having all these function calls in between makes it all much easier to read.

If that is too basic for you, maybe something like what the C++ generator does, with overloaded += etc.. but not a fan of that either.

[dbaileychess]

Noted. I'll moved out of base class.

[dbaileychess]

I do prefer the more verbose style, especially if performance is not an issue. Which it isn't during code gen. The compiler is just going to inline all the method calls anyways.

[dbaileychess]

I'll leave it for now and we can refactor after this is in.

[CasperN]

While we're bike shedding, my 2c is that I like the += thing. I also try to use methods like ForAllFieldsInTable or ForAllVariantsInEnum which take a lambda and handles indentation. This is neat since the code in the lambda will be indented anyway. If the ForAllX method did not handle indentation, then the raw string would seem double indented. This is of course pretty far on the DRY/overengineered side of the spectrum

[dbaileychess]

Are you suggesting placing ForAllFieldsInTable and ForAllVariantsInEnum in the base class and then the implementing classes just provide a custom lambda?

[dbaileychess]

I went ahead and refactored it all to use the code += style of code gen.

I took @CasperN idea to define those functions that took a lambda, as that hid away the details of how we were looping over the vectors.

[dbaileychess]

I ended up replacing the old --lua generator with this new one by default. I'll remove the old code in a follow up CL so it is easier to revert if needed.