falsely deserialization for big numbers

[LiranBri]

Hi, I noticed that big numbers are being falsely deserialized. Is there a workaround?

code to reproduce:

const RoaringBitmap32 = require('roaring/RoaringBitmap32');

itemIds = [4858410435, 100]
itemIdsAsBitmap = new RoaringBitmap32(itemIds)

serialized = itemIdsAsBitmap.serialize(false).toString('base64')
deserialized = RoaringBitmap32.deserialize(Buffer.from(serialized, 'base64'), false).toArray();
console.log(deserialized) // prints [ 100, 563443139 ]

[LiranBri]

I realized now that the lib only supports 32 bits numbers. that unfortunate :(

[SalvatorePreviti]

Is called RoaringBitmap32 for a reason. 32 bit unsigned integers. Roaring bitmap data structure is 32 bits by design. CRoaring C++ version supports a type of bitmap that can store 64 bits by creating up to 4 billion bitmaps for the higher 32 bits that sincerely seems not a very good idea to use in most circumstances (out of memory very easy to cause) and so is not implemented in the NodeJs version especially because Js numbers are 53 bit wide (that's the biggest safe integer that can be stored in a double floating point value) and using bigint would be so slow that the performance improvements provided by the roaring data structure would not give any benefit

[SalvatorePreviti]

If in your specific case you have just few bits more than 32 to store you can have more than one bitmap and split the number in higher bits, to select the right bitmap, and lower bits, to use in the selected bitmap. Is not going to be fun because bit operators in JavaScript works only on the lowest 32 bit, they are int32, so you will have to use floating point magic: division and truncation instead of shift left or &. And as said before, you cannot store all possible 64 bit integers in a JavaScript number, it can store maximum 53 bit integers. There is BigInt for bigger integers

[LiranBri]

thanks @SalvatorePreviti