-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: discard txs by tx_count of sender #4520
Conversation
Codecov Report
Additional details and impacted files
... and 442 files with indirect coverage changes
Flags with carried forward coverage won't be shown. Click here to find out more.
|
reviewing this and will probably be touching some of it up soon! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice, I think we're almost there.
making them pub should be fine, I can see the pool structs being useful
@@ -243,6 +246,23 @@ impl<T: TransactionOrdering> PendingPool<T> { | |||
removed | |||
} | |||
|
|||
/// Updates the independent transaction and independent descendants set, assuming the given |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
needs reference "highest nonce" instead
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added
let mut removed = Vec::new(); | ||
|
||
// penalize non-local txs if limit is still exceeded | ||
while self.size() > limit.max_size || self.len() > limit.max_txs { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is the inverse of L360 and L380?
I think we can rewrite this as loop {}
and only keep a break
let mut any_non_local = true; | ||
|
||
// use descendants, so we pop a single tx per sender | ||
for tx in self.highest_nonces.clone().iter() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
even though this only includes unique senders and we expect that the outer loop is really a single iterator.
I think we get rid of the clone if we instead push all txs ids that need to be removed to a tmp vec, keep track of the amount and sizes and only remove them once we have collected enough, then do the same again without locals check if necessary
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
tried to implement this with transactions_to_remove
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
great,
I only have one suggestion left for the pending eviction routine
|
||
// index of first tx to check | ||
let mut removed_idx = 0; | ||
'outer: loop { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not immediately obvious why we need this and this looks a bit complex.
I guess we need this because removing 1 tx per unique sender might not be sufficient? But the highest nonce is only updated once we remove the transactions?
I think this can be simplified by putting L372-L397 in a loop that actually removes the recorded ids of removed
so we don't need the loop below?
If the function takes a &mut Vec
then the transactions can be pushed directly to this, for both (locals =true|false) calls
Under real circumstances this will only evict 1 to a few transactions so having a tmp buffer for the removed
ids isn't too bad.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
great,
only cleanup then I believe this is done
total_size += tx.transaction.size(); | ||
removed.push(*tx.transaction.id()); | ||
} | ||
// we can re-use the temp array |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice
/// If the `remove_locals` flag is unset, then only non-local transactions will be removed. | ||
pub(crate) fn transactions_to_remove( | ||
&self, | ||
pub(crate) fn limit_pool( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
name should convey that this removes
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
renamed to remove_to_limit
removed.push(tx); | ||
} | ||
} | ||
self.limit_pool(&limit, true, &mut removed); | ||
|
||
removed.shrink_to_fit(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do we need to shrink anything here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nope, was left over from when the map was created with_capacity
, removed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
great.
I'm very happy with how this turned out
.into_iter() | ||
// sort by submission id | ||
.collect::<BinaryHeap<_>>(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not exactly sure how this works internally but I assume this into_iter collect is optimized so there are no allocs,
but I think this could be written as a loop over an iterator that pushes unique senders directly into a binaryheap
Closes #4269