Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

s3_lifecycle - Add size thresholds #861

Closed
1 task done
mdavis-xyz opened this issue Jan 13, 2022 · 3 comments · Fixed by #2205
Closed
1 task done

s3_lifecycle - Add size thresholds #861

mdavis-xyz opened this issue Jan 13, 2022 · 3 comments · Fixed by #2205
Labels
feature This issue/PR relates to a feature request good first issue waiting_on_contributor Needs help. Feel free to engage to get things unblocked

Comments

@mdavis-xyz
Copy link
Contributor

Summary

STANDARD_IA and GLACIER classes in S3 include an overhead for access history data. For small files the overhead can be larger than the file itself, so you end up paying more to store small files in these supposedly cheaper classes.

The solution is to add a minimum size filter to the lifecycle config. The API supports this. The s3_lifecycle ansible module does not yet support these. We should add a min size filter (and also max size filter).

Issue Type

Feature Idea

Component Name

s3_lifecycle

Additional Information

In the boto3 call, ObjectSizeGreaterThan and ObjectSizeLessThan are optional fields which go next to Prefix under Filter. They are the number of bytes (not kB).

I propose this usage:

- name: Configure a lifecycle rule on a bucket to expire (delete) items with a prefix of /logs/ after 30 days
  community.aws.s3_lifecycle:
    name: mybucket
    expiration_days: 30
    prefix: logs/
    object_size_greater_than: 123
    object_size_less_than: 123
    status: enabled
    state: present

Code of Conduct

  • I agree to follow the Ansible Code of Conduct
@markuman markuman added feature This issue/PR relates to a feature request good first issue labels Jan 13, 2022
@alinabuzachis
Copy link
Contributor

@mdavis-xyz Thank you for this feature idea. Would you be willing to work on it and open a pull request?

@mdavis-xyz
Copy link
Contributor Author

I just found out that Lifecycles actually already have a minimum size filter for these transitions.

link to docs

Objects smaller than 128 KB – For the following transitions, Amazon S3 does not transition objects that are smaller than 128 KB:

From the S3 Standard or S3 Standard-IA storage classes to S3 Intelligent-Tiering or S3 Glacier Instant Retrieval.

From the S3 Standard storage class to S3 Standard-IA or S3 One Zone-IA.

Therefore I don't need this new Ansible functionality for my use case. So I probably won't write this until much later this year when I get some spare time (and after writing other PRs I more urgently need.)

@abikouo abikouo added the waiting_on_contributor Needs help. Feel free to engage to get things unblocked label Jan 20, 2022
@tremble tremble changed the title Add size thresholds to s3_lifecycle s3_lifecycle - Add size thresholds Feb 6, 2023
@leo4ever leo4ever mentioned this issue Dec 17, 2024
Merged
@leo4ever
Copy link
Contributor

@alinabuzachis - I have raised PR #2205 to include minimum and maximum size thresholds.

patchback bot pushed a commit that referenced this issue Dec 29, 2024
@leo4ever @patchback
Support min max object size in s3_lifecycle module (#2205)
e28d437 
SUMMARY

Support the S3 lifecycle settings of minimum and maximum object size to apply the lifecycle rules.

Fixes #861
ISSUE TYPE

Feature Pull Request

COMPONENT NAME

s3_lifecycle
ADDITIONAL INFORMATION

Reviewed-by: Mark Chappell
Reviewed-by: Javid Alimohideen
(cherry picked from commit af56b7f)
@patchback patchback bot mentioned this issue Dec 29, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature This issue/PR relates to a feature request good first issue waiting_on_contributor Needs help. Feel free to engage to get things unblocked
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Support min max object size in s3_lifecycle module leo4ever/community.aws
5 participants
@leo4ever @markuman @mdavis-xyz @alinabuzachis @abikouo