Dynamic Memory Usage in KDL Parsers

[garrisonhh]

In languages like ruby, python, java, rust etc. using dynamic memory (things like ArrayLists and infinitely growable strings) is ubiquitous and expected. The current KDL spec reflects this, and doesn't define character limits on things like string literals or the number of child nodes a node is expected to allow. In order to be completely spec conforming, this means that KDL libraries must use dynamic memory allocation. If I wanted to use KDL without an operating system, or write a program that parses KDL without stopping to perform syscalls (which performance-wise is pretty critical), I'm forced to go non-compliant.

So I propose that maximum lengths for arguments, properties, children-per-node, identifiers, and strings be added to the spec, even in an optional capacity!

[djmattyg007]

Does anything like this exist for JSON, XML, etc?

[hkolbeck]

After a little research, it seems that other similar document languages leave maximums out of the spec and instead allow implementations to define their own, hopefully configurable. I think a similar approach is most desirable for KDL.

[tabatkins]

Agreed, implementation/environment-specific limitations are completely reasonable, but should be specified by the impl/environment; they don't need to be built into the language unless the language has some special connection to such an environment (intentionally designed to be performant there, only really expected to be used in such environments, etc)

[nichtich]

In order to be completely spec conforming, this means that KDL libraries must use dynamic memory allocation.

If you care about memory allocation and parsing performance please have a look at simdjson, the fastest JSON parser.

I'm forced to go non-compliant.

You can avoid copying any strings if you only store the start-position and length of strings and identifiers in the input stream. Your limit (there is always a technical limit) is then moved from input length to the number KDL of elements in your input, so you gain an order of magnitude.