Skip to content

Commit

Permalink
handle trailing bytes in bitpacked run
Browse files Browse the repository at this point in the history
There may be trailing bytes in bitpack encoded data. The reader should be able to skip those while reading bit packed runs.

fixes #120
  • Loading branch information
tanmaykm committed Dec 3, 2020
1 parent 5865a32 commit 4f54dfc
Showing 1 changed file with 8 additions and 2 deletions.
10 changes: 8 additions & 2 deletions src/codec.jl
Original file line number Diff line number Diff line change
Expand Up @@ -294,11 +294,17 @@ function read_hybrid(inp::InputState, out::OutputState{T}, count::Int32, bits::U
runhdr = _read_varint(inp, Int)
isbitpack = ((runhdr & 0x1) == 0x1)
runhdr >>= 1
nitems = min(isbitpack ? runhdr*8 : runhdr, count - arrpos + 1)
nrunhdrbits = runhdr * 8
nitems = min(isbitpack ? nrunhdrbits : runhdr, count - arrpos + 1)

if isbitpack
runcount = min(runhdr * 8, length(arr)-offset-arrpos)
runcount = min(nrunhdrbits, length(arr)-offset-arrpos)
read_bitpacked_run(inp, out, runcount, bits, byt, mask)
if nrunhdrbits > runcount
nbytes_to_skip = div(nrunhdrbits - runcount, 8)
@debug("skipping trailing bytes in bitpacked run", nbytes_to_skip)
inp.offset += nbytes_to_skip
end
#out.offset += runcount
else # rle
read_type = @byt2uitype(byt)
Expand Down

0 comments on commit 4f54dfc

Please sign in to comment.