Skip to content

Commit

Permalink
Avoid infinite loop in bad parquet by checking the number of rep leve…
Browse files Browse the repository at this point in the history
…ls (#6232)

* check the number of rep levels read from page

* minor fix on typo

Co-authored-by: Andrew Lamb <[email protected]>

* add check on record_read as well

---------

Co-authored-by: jp0317 <[email protected]>
Co-authored-by: Andrew Lamb <[email protected]>
  • Loading branch information
3 people authored Aug 15, 2024
1 parent c835f88 commit 042d725
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions parquet/src/column/reader.rs
Original file line number Diff line number Diff line change
Expand Up @@ -240,6 +240,12 @@ where
let (mut records_read, levels_read) =
reader.read_rep_levels(out, remaining_records, remaining_levels)?;

if records_read == 0 && levels_read == 0 {
// The fact that we're still looping implies there must be some levels to read.
return Err(general_err!(
"Insufficient repetition levels read from column"
));
}
if levels_read == remaining_levels && self.has_record_delimiter {
// Reached end of page, which implies records_read < remaining_records
// as otherwise would have stopped reading before reaching the end
Expand Down

0 comments on commit 042d725

Please sign in to comment.