Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reevaluate usage of AWS SDK #1479

Closed
Swatinem opened this issue Jun 7, 2024 · 2 comments
Closed

Reevaluate usage of AWS SDK #1479

Swatinem opened this issue Jun 7, 2024 · 2 comments

Comments

@Swatinem
Copy link
Member

Swatinem commented Jun 7, 2024

We should figure out what we want to do with the AWS SDK going forward.

Right now, it is the single biggest offender related to compile times, and pulls in a huge swath of dependencies.
Its so bad I dedicated a blog post to it: https://swatinem.de/blog/bloaty-mcbloat-sdk/
Which itself reminded me about a talk I gave ~4½ years ago about equally bad bloat in their JS SDK: https://viennajs.org/en/meetup/2020-01/optimizing-nodejs-memory-usage

But it is not only about it being horrendously bloated. They also break the API, and especially their error handling all the time.
I had to work around one of those issues here:

// work around https://github.com/awslabs/aws-sdk-rust/issues/1148#issuecomment-2124123894:
// we seem to get a bogus `ContentLengthError` because *obviously*
// the `HEAD` request returns an empty body while another header reports an expected `content-length`.
_ if format!("{err:?}").contains("ContentLengthError") => {}

And a recent upgrade has again introduced a ton of fresh errors that were previously handled "correctly".

I’m honestly not sure what to do about this exactly. We do want to use uptodate and maintained crates, however you define "maintained".
But the amount of hassle we have with the AWS SDK is in no relation to our usage. We only want to download files, nothing more nothing less, we don’t need the whole kitchen sink.

CC @josb just curious to hear your feedback on this, given you contributed the initial rewrite from rusoto to the official SDK.

@josb
Copy link
Contributor

josb commented Jun 7, 2024

@Swatinem my first instinct is to make it clear to AWS that these are also issues for this project. The compile time issue is definitely on their radar.

A quick search for a Rust S3 client only shows https://github.com/durch/rust-s3 but it looks unmaintained.

@loewenheim
Copy link
Contributor

Closing as not actionable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

No branches or pull requests

3 participants