Skip to content

Commit

Permalink
Assume malformed head chunk is partially written data
Browse files Browse the repository at this point in the history
Head chunks and the WAL may not be completely flushed to disk if
e.g., Prometheus crashes.

Partial records may exist at the end of the file; allow this and assume
there is not further valid data.

A future change should validate CRCs also, and stop reading on the first
incorrect CRC.
  • Loading branch information
jameseh96 committed Aug 14, 2023
1 parent 92973f6 commit 4a774f4
Showing 1 changed file with 9 additions and 3 deletions.
12 changes: 9 additions & 3 deletions src/pdu/block/chunk_reference.cc
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,15 @@ std::optional<std::pair<size_t, ChunkReference>> readHeadChunkMeta(

size_t dataLen = dec.read_varuint();

// skip data and 4 byte crc, leaving decoder at the start of the next
// chunk meta header.
dec.seek(dataLen + 4, std::ios_base::cur);
try {
// skip data and 4 byte crc, leaving decoder at the start of the next
// chunk meta header.
dec.seek(dataLen + 4, std::ios_base::cur);
} catch (const std::exception&) {
// if we fail to seek that far, assume that this file was partially
// written. There should be no later valid entries
return {};
}

return {{seriesRef, std::move(ref)}};
}

0 comments on commit 4a774f4

Please sign in to comment.