Parser development workflow hindered by slow compile/JIT

[jsiwek]

This may be deserving of a formal performance/usability issue, but just throwing out observations from my first Spicy parser development experience of a PE parser: https://github.com/zeek/spicy/compare/topic/jsiwek/pe

Even as I started out with a basic parsing structure, I noticed slow iteration time with compile/JIT taking ~10-15 seconds, and by the end, the Spicy grammar of only ~500 lines is taking 30+ seconds. Doesn't ultimately impact functionality of the parser, but seems significant in terms of scalable development process: long turnaround times make initial learning of Spicy difficult and likely also slow development of even more complex specifications.

Robin mentioned it's may be possible to look into whether particular C++ constructs in backend generated code could be replaced w/ C constructs that compilers may digest more quickly. Whether it's that or another idea, think the general usability issue is worth some attention.

[rsmmr]

Context for others reading this: there's a profiling option -R in spicy-driver/spicyc/spicyz that reports where time is spent during compilation. The cases I looked at most recently spent ~90% in compiling the generated C++ code, so speeding up that would yield the most benefit (we did some low-hanging fruit there already, but clearly it's still taking surprisingly long).

I believe we also have some grammars where Spicy itself needs more than 10%, that's something to look into as well, should be fixable.

[rsmmr]

The work in https://github.com/zeek/spicy/wiki/Project:-Improve-efficiency-of-the-generated-C-----parsing-code/ will help with this.