Gob encoding and internal data types

[ostafen]

Hi, everyone, I created this discussion to collect opinions and suggestions, since this is a very sensitive topic.
Currently, CloverDB serializes documents to json before storing them on disk. This has been done because of the fact that, early versions of the library used ".json" files directly to store data.
But since Clover evolved since that time (it now uses the badger kv-store), this solution is no more acceptable for the following reasons:

• instances of the time.Time struct cannot be correctly recovered, because they are converted to string when serialized and, as a consequence, json.Unmarshal() deserializes them to normal stings. This affects queries involving dates or times (unless you decide to store them as a timestamp during document insertion).
• All numbers are silently converted to float64.

To fix these issues, I was thinking to switch to the gob encoding, which preserves the correct type for each document field.
This open a new question about internal data types:

Which numeric types should be supported by clover? Should we preserve all of the types (int, uint8/int8, uint16/int16... and so on) or should we restrict types (using int64 for integer numbers and float64 for double numbers, for example).

What do you think about this?

[segfault99]

Hello, i couldn't pay attention because i was busy last few days.
In my opinion using int64 and float64 would make the interface cleaner and implementing process easier.

[ostafen]

Thank you for your feedback :=)
What do you think of including uint64, also?

[segfault99]

Yes, including uint64 also could be useful.

[ASWLaunchs]

The GoB package is a "private" codec provided by Golang.They say it's efficiency will be higher than JSON, XML, etc. Therefore, the communication between the two Go services like chat app, log app always pursure time accuracy, this kind of data consistency is useful for developing instant messaging products to calibrate time. so the GoB might is a good solution. 👍
But on other hand , I don’t know if there are other side effects.

[ostafen]

I created PR #47 for this :=)