Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compress the json files while uploading/downloading from S3 #218

Open
sara-02 opened this issue May 18, 2018 · 1 comment
Open

Compress the json files while uploading/downloading from S3 #218

sara-02 opened this issue May 18, 2018 · 1 comment

Comments

@sara-02
Copy link
Contributor

sara-02 commented May 18, 2018

Since these files are downloaded everytime the service is brought up, it incurs a huge amount of data to be transferred in terms of the JSON files of the model. Compressing them will reduce the amount of data transferred.
As pointed out by @msrb

$ ls -lh kronos_dependency_maven.*
-rw-rw-r--. 1 msrb msrb  51M May 16 14:01 kronos_dependency_maven.json
-rw-rw-r--. 1 msrb msrb 6.2M May 16 14:02 kronos_dependency_maven.zip

@CermakM
Copy link
Contributor

CermakM commented May 22, 2018

We might consider using Protocol Buffers for this ... here is a comparison of size w.r.t. JSON.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants