You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently SparseCompressor is disabled for most input types. It will ONLY accept another SparseArray, and will re-compress the patches.
Our SparseArray is more or less identical to the Frequency encoding from btrblocks (which is a simplified version of Frequency encoding from DB2). As an example of somewhere it would be useful to compress using the Sparse array, consider the following compression pathway we use currently in ClickBench for an i32 array with 999,777 instances of -1, and 223 instances of some 103 unique values:
While I don't have a solid benchmark, I can imagine this is causing a lot of extraneous overhead on the read pathway.
The text was updated successfully, but these errors were encountered:
Currently SparseCompressor is disabled for most input types. It will ONLY accept another SparseArray, and will re-compress the patches.
Our SparseArray is more or less identical to the Frequency encoding from btrblocks (which is a simplified version of Frequency encoding from DB2). As an example of somewhere it would be useful to compress using the Sparse array, consider the following compression pathway we use currently in ClickBench for an i32 array with 999,777 instances of
-1
, and 223 instances of some 103 unique values:While I don't have a solid benchmark, I can imagine this is causing a lot of extraneous overhead on the read pathway.
The text was updated successfully, but these errors were encountered: