Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid intermediate JSON objects when serializing and deserializing #662

Open
osa1 opened this issue May 25, 2022 · 0 comments · May be fixed by #670
Open

Avoid intermediate JSON objects when serializing and deserializing #662

osa1 opened this issue May 25, 2022 · 0 comments · May be fixed by #670
Labels
perf Related to runtime performance

Comments

@osa1
Copy link
Member

osa1 commented May 25, 2022

Currently methods like mergeFromProto3Json that parse a message encoded as JSON take Dart representation of the JSON message as argument. This means to parse a JSON message we first parse the JSON to a Dart object representing the JSON, then convert it to a message.

We could avoid the intermediate step of parsing JSON to Dart maps and lists by using a "pull based" JSON parser like jsontool.

I don't know if it's common to pass JSON objects around within the same Dart app (without serializing JSON objects) and update messages with them, but if this is common we could keep the current methods as well.


A concern when targeting JS is that we use JS engine's JSON.parse to parse JSON to JS representation of the JSON, and then convert it to Dart representation. So conversion of JSON serialization to JS objects is done by the engine and given how common JSON is I suspect this code is highly optimized. So on the JS we will be comparing:

  1. Parsing JSON string to JSON (done by the JS engine, probably highly optimized) + converting JSON object to a Dart object (done by Dart code, in core) + updating message fields using the Dart (in protobuf library)
  2. Updating message fields using the pull-based parser (entirely in protobuf library)

It is possible that parsing JSON string in Dart is so much slower than parsing JSON string to JSON object at the engine level, the pull-based approach will be slower on the browser. Also, it seems like when targeting JS, generating of Dart objects from a JSON object is done lazily (_convertJsonToDartLazy in the VM code linked above), so perhaps the GC pressure is also not high in (1).

With AOT and JIT, the only help we get from the VM is Double_parse for parsing doubles. Rest of the JSON parsing seems to be in pure Dart, and as far as I can see we don't have the lazy conversion that we have in JS version, so I suspect using a pull-based parser will probably a win when targeting JIT and AOT.

We could use conditional compilation and fall back to the current implementation if this only performs well in AOT/JIT and not in JS.


Similarly for serializing as JSON we could use jsontool's JsonStringWriter and avoid intermediate JSON objects.

We already have benchmarks for parsing and generating JSON, so evaluation shouldn't be too difficult.

@osa1 osa1 added the perf Related to runtime performance label May 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
perf Related to runtime performance
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant