-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(commands): add resume on error for comments and emails #291
Conversation
api/src/lib.rs
Outdated
Method::PUT, | ||
self.endpoints.put_comments(source_name)?, | ||
PutCommentsRequest { | ||
comments: comments.to_vec(), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because you're taking the input comments
and just converting it to Vec
, it would be better to declare these functions as taking a Vec
instead of a slice.
api/src/lib.rs
Outdated
debug!("Attempting {} `{}`", method, url); | ||
let result: Result<SuccessT> = self.request( | ||
method.clone(), | ||
url.clone(), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some of this cloning looks as if it should be unnecessary, e.g. IntoUrl
is implemented for &str
.
match result { | ||
Ok(response) => Ok(SplitableRequestResponse { | ||
response, | ||
num_failed: 0, | ||
}), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be nice not to have to write the main body of code for this function nested inside a match
branch and a conditional. Something like:
match result {
Ok(response) => return Ok(SplitableRequestResponse {
response,
num_failed: 0,
}),
Err(error) if !Self::should_split(&error) => return Err(error),
_ => {},
}
...
api/src/lib.rs
Outdated
@@ -1250,6 +1401,7 @@ impl Client { | |||
} | |||
} | |||
|
|||
#[derive(Clone)] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For small structs and enums (those with size <= 8 bytes, say), Copy
should also be derived, as they're cheap to copy. Then clone()
ing them is unnecessary.
api/src/resources/comment.rs
Outdated
Self { | ||
new: a_new + b_new, | ||
updated: a_updated + b_updated, | ||
unchanged: a_unchanged + b_unchanged, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just write this as a.new + b.new
etc. and omit let
bindings?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added the let binding so that we would get a compiler error if new fields were added (to remind us to make any necessary changes here)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rather than do that, you can just put a single statement like this at the top:
// Ensure new fields are added here:
let Self { new: _, updated: _, unchanged: _ } = a;
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks - just pushed a slightly different implementation
api/src/resources/comment.rs
Outdated
fn empty() -> Self { | ||
Self {} | ||
} | ||
fn merge(_a: Self, _b: Self) -> Self { | ||
Self {} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can supply a default implementation for trait methods in the trait itself that will be used if there's no implementation for the type. This should allow you to avoid repeating these trivial implementations (you'll need a Default
trait bound).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I chose not to use the Default
trait initially as it doesn't make sense to have Default
for these responses unless you're using the ReducableResponse
trait. Don't have strong opinions here though
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In that case you can make your own version of Default
called ReducibleDefault
or whatever.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Implemented with Default
for the sake of simplicity :)
api/src/lib.rs
Outdated
fn should_split(error: &Error) -> bool { | ||
if let Error::Api { status_code, .. } = error { | ||
*status_code == reqwest::StatusCode::UNPROCESSABLE_ENTITY | ||
|| *status_code == reqwest::StatusCode::BAD_REQUEST | ||
} else { | ||
false | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can define this within splittable_request
, as it's a helper private to the method.
api/src/lib.rs
Outdated
pub num_failed: usize, | ||
} | ||
|
||
pub trait ReducableResponse { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
pub trait ReducableResponse { | |
pub trait ReducibleResponse { |
api/src/lib.rs
Outdated
query.clone(), | ||
retry.clone(), | ||
) { | ||
Ok(r) => response = SuccessT::merge(response.clone(), r), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, this mutation and cloning doesn't look quite right. Conceptually we should be able to reduce
the successful responses, although we need to filter out and handle the failures somehow.
api/src/lib.rs
Outdated
|
||
let response = SuccessT::reduce( | ||
body.split() | ||
.iter() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is looking much better, but we should be able to do better still as this is a standard fold pattern, if you switch back to the merge
trait method:
let response = body
.split()
.iter()
.filter_map(|request| {
match self.request(&method, &url, &Some(request), &query, &retry) {
Ok(response) => Some(response),
Err(_) => {
num_failed += 1;
None
}
}
})
.fold(SuccessT::empty(), |merged, next: SuccessT| merged.merge(next));
api/src/lib.rs
Outdated
@@ -137,6 +143,35 @@ pub use crate::{ | |||
#[derive(Clone, Debug, PartialEq, Eq)] | |||
pub struct Token(pub String); | |||
|
|||
pub trait SplittableRequest { | |||
fn split(self) -> Vec<Self> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, you should be able to define split
as returning an impl Iterator
instead of a Vec
and remove the collect()
at the end of the implementations.
bdc98a6
to
29b7411
Compare
29b7411
to
ce12437
Compare
Adds a
--resume-on-error
flag for uploading comments and emails. When the api returns an error the CLI will split a particular batch into individual requests, attempt each one individually and skip any individual request that fail