Parallelize S3 delete objects requests #5535
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
trinity-1686a
approved these changes
Nov 4, 2024
|
|
||
| impl CancellationToken { | ||
| fn cancel(&self) { | ||
| self.0.store(true, Ordering::Release); |
There was a problem hiding this comment.
this access can be Relaxed, we're only interested in how the memory address changes over time, not to how its changes relate to changes in the rest of the memory of our program
| } | ||
|
|
||
| fn is_cancelled(&self) -> bool { | ||
| self.0.load(Ordering::Acquire) |
Comment on lines
+625
to
+627
| let bucket = self.bucket.clone(); | ||
| let retry_params = self.retry_params; | ||
| let cancellation_token = cancellation_token.clone(); |
There was a problem hiding this comment.
because of how this code is written (structured concurrency vs the more usual tokio::spawn), bucket and cancellation_token need not be cloned, refs are enough (and CancellationToken could be made to not have an inner Arc)
i don't know if it should be done that way, but it certainly can
|
This PR is no longer necessary and since the patch is complex, I'd rather close. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Parallelize S3 delete objects requests
How was this PR tested?
Ran
quickwit-storagetest suite locally.