Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disallow compactions into L5/L6 for files that straddle a snapshot stripe #2463

Closed
itsbilal opened this issue Apr 18, 2023 · 2 comments
Closed

Comments

@itsbilal
Copy link
Member

Currently, sstables in L5 and L6 can contain multiple internal keys for a user key (eg. a del and a set), and even range deletes and range key sets/unsets/dels that haven't been coalesced even within the sstable. This is because of the presence of snapshots at the time of sstable creation. If we can guarantee that sstables in L5 and L6 were all on one side of a snapshot stripe at time of creation, we can greatly simplify some disaggregated storage read-time coalescing (see #2455), at the potential expense of preventing some L4 -> L5 compactions from happening and resulting in more sstables accumulating in L4. That might not be an issue if snapshots are rarely (and decreasingly) used.

Implementation for this should be relatively straightforward, with the compaction picker excluding any files that have a [SmallestSeqNum, LargestSeqNum] overlapping with an open snapshot seqnum, if the compaction is bound for L5/L6.

@sumeerbhola
Copy link
Collaborator

Is this issue obsolete given #2465?

@itsbilal
Copy link
Member Author

It's a lot less necessary given that, yes. We still do read-time coalescing for range keys (see ForeignSSTTransformer), and due to EFOS we write less obsolete keys to begin with. Closing this off as the priority of it is low enough to make it not worthwhile doing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

No branches or pull requests

2 participants