switch s3 putobject to upload function #2801
Conversation
Signed-off-by: Jonathan Holloway <[email protected]>
loadtheaccumulator
force-pushed
the
fix_upload_timeouts
branch
from
54d9b53 to
730426f
Compare
There was a problem hiding this comment.
Yep, this is exactly what the documentation says. Upload function uses MIME multipart upload which can likely resurrect failed transaction.
https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpu-upload-object.html
|
Docs are a wonderful thing. :-) |
|
One remark tho, my guts tells me that Upload will be more resource heavy than PutObject and using it for ALL uploads might slow down upload of small files. |
|
I don't think it will be huge difference, but do we have any telemetry around this to see potential spike once we merge? |
This was just a warning, I think it will be okay but it could possibly worsen uploads for many small files (repo upload). Just keep an eye on this @loadtheaccumulator no action is needed at this point. |
|
NOTE: It can also be tuned via the Uploader. |
Description
Repo file uploads occasionally timeout--especially on slower upstream connections.
Switching from PutObject() to Upload() resolves the issue.
FIXES: HMS-5534
Type of change
What is it?