Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Json writer for cp2077-nativedb #7

Merged
merged 3 commits into from
Feb 6, 2024

Conversation

poirierlouis
Copy link
Contributor

Custom AST Json writer to be used with webapp cp2077-nativedb.

…abase.

Writer create individual files per definition.
Writer create grouped files per type of definitions (enums/bitfields/classes/globals).
Writer use an obfuscated Json schema to minimize memory footprint (even better when compressed with Brotli).
Writer only include required fields.
Writer reformat and compress flags with only known and reused data.
Writer is to be used by webapp 'cp2077-nativedb'.
…abase.

Writer create individual files per definition.
Writer create grouped files per type of definitions (enums/bitfields/classes/globals).
Writer use an obfuscated Json schema to minimize memory footprint (even better when compressed with Brotli).
Writer only include required fields.
Writer reformat and compress flags with only known and reused data.
Writer is to be used by webapp 'cp2077-nativedb'.
@wopss
Copy link
Owner

wopss commented Feb 6, 2024

Thanks! Overall looks good but I have few questions:

  1. Could you please tell me the difference between this and JsonWriter.hpp?
  2. I see that you use one letter for the AST, could you explain why one letters instead of the fully name?

@poirierlouis
Copy link
Contributor Author

poirierlouis commented Feb 6, 2024

1. Could you please tell me the difference between this and [JsonWriter.hpp](https://github.com/WopsS/RED4.RTTIDumper/blob/master/src/Writers/JsonWriter.hpp)?

Flags are "compressed" such as only known bits / used bits in web app are stored in Json. It writes primitive types / template types as numbers too. It also writes definitions in grouped files (enums.json / bitfields.json / classes.json / globals.json) so only a few files are requested in web app. Fullname / shortname is only stored when there is a difference for example. The main purpose is to make Json data as compressed as possible to minimize bandwidth usage.

2. I see that you use one letter for the AST, could you explain why one letters instead of the fully name?

I believe that reusing identical names improves compression ratio, as it should provide a minimum of different symbols, resulting in a smaller memory footprint (e.g. 10MB to 800KB with Brotli).

@wopss
Copy link
Owner

wopss commented Feb 6, 2024

Ah, I see. Thank you for the explanation!

@wopss wopss merged commit 96b0c7e into wopss:master Feb 6, 2024
2 checks passed
@poirierlouis poirierlouis deleted the nativedb-json-writer branch February 6, 2024 14:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants