tip for performance

[ejamshidiasl]

i sent a post 10 min ago and i remembered something else from one of my old projects.

storing data in json format cause increasing size of file and memory. there is a better way:

we need 2 lists:
list#1 store column_headers (name, age, data1, etc)
list#2 storing data without headers. then on search, update, etc. we can convert user input (name, age, etc.) to index in list#2 with using list#1

my english is not good:D

simple example:
database: users database
list#1 (programmer defiend headers) => list_1 = ('name','age','tel','address')
list#2 (data) => list_2 = [ ['name1','age1','tel1','address1'] , ['name2','age2','tel2','address2'] , ... ]

so we just store headers once and not thousonds of times!

sotring in file: NOT JSON
better way simple example: users.db (first line headers, after that data each line is one row of data)

name,age,tel,address
id1,name1,age1,tel1,address1
id2,name2,age2,tel2,address2

[msiemens]

You're free to create a new storage that has the performance characteristics you like best 🙂 Using JSON to store data has both upsides and drawbacks. As you mentioned the drawback is that storing lots of similar documents will result in overhead due to the duplicated JSON keys. The upside however is that it gives great flexibility when different documents have different keys. In your example, adding a new document like {"name": "foo", "age": 12, "tel": 123, "address": "something", "food": "bread"} would require all previous rows to be rewritten to account for the new column. So it's trade-offs all the way down 🙂