Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

s3_logging add bucket policy as default access method to target bucket #2108

Closed

Conversation

abraverm
Copy link

SUMMARY

S3 ACL is disabled by default and AWS documentation encourages using policies.
This change adds policy as the default option and allows switching between ACL and policy.

https://docs.aws.amazon.com/AmazonS3/latest/userguide/enable-server-access-logging.html

ISSUE TYPE
  • Feature Pull Request
COMPONENT NAME

s3_logging

ADDITIONAL INFORMATION

The module modifies target bucket policy.
Reference:
https://docs.aws.amazon.com/AmazonS3/latest/userguide/enable-server-access-logging.html

https://docs.aws.amazon.com/AmazonS3/latest/userguide/about-object-ownership.html#object-ownership-setting

A majority of modern use cases in Amazon S3 no longer require the use of ACLs, and we recommend that you keep ACLs disabled except in unusual circumstances where you must control access for each object individually. With ACLs disabled, you can use policies to more easily control access to every object in your bucket, regardless of who uploaded the objects in your bucket.

Copy link
Contributor

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/23edf143331a40399aaeeaf6b15e96c4

✔️ ansible-galaxy-importer SUCCESS in 3m 52s (non-voting)
✔️ build-ansible-collection SUCCESS in 12m 28s
✔️ ansible-test-splitter SUCCESS in 4m 57s
✔️ integration-community.aws-1 SUCCESS in 6m 02s
Skipped 21 jobs

@abraverm abraverm force-pushed the s3_logging_policy branch 2 times, most recently from caaa494 to bfb458f Compare June 26, 2024 13:41
Copy link
Contributor

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/dbd6c25a592e482da47d778d61b83aca

✔️ ansible-galaxy-importer SUCCESS in 3m 28s (non-voting)
✔️ build-ansible-collection SUCCESS in 12m 32s
✔️ ansible-test-splitter SUCCESS in 5m 03s
✔️ integration-community.aws-1 SUCCESS in 5m 12s
Skipped 21 jobs

@hakbailey hakbailey modified the milestones: 8.1.0, 9.0.0 Jun 26, 2024
@abraverm abraverm force-pushed the s3_logging_policy branch from bfb458f to 414da1f Compare July 18, 2024 14:57
Copy link
Contributor

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/60218c4fa8594e099d8a9e16db689a16

✔️ ansible-galaxy-importer SUCCESS in 3m 31s (non-voting)
✔️ build-ansible-collection SUCCESS in 12m 56s
✔️ ansible-test-splitter SUCCESS in 5m 05s
✔️ integration-community.aws-1 SUCCESS in 7m 36s
Skipped 21 jobs

@abraverm
Copy link
Author

Hi @tremble and @markuman, could you review my PR please?

Copy link
Contributor

@tremble tremble left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for taking the time to submit this PR.

Unfortunately, as it stands today I would be a -1 on this PR:

  1. I don't see the need for this to be implemented as a breaking change

  2. This kind of automatic manipulation of policies is inherently unstable.
    For example if someone already used that Sid in the bucket policy then the update will fail.

  3. It looks like it may still attempt to manipulate the bucket policy even if someone tries to continue using ACL based access

  4. The change as written requires additional bucket permissions even if using the old ACL method rather than your new policy based mechanism

What I would be more open to is a mechanism for disabling the setting of the ACL, so the someone can manage the policy directly. We already support managing access policies with amazon.aws.s3_bucket.

I suspect that you're trying to mimic some of the AWS Web-UI features, however, policies are very finicky and trying to do this in Ansible just leads to people opening bugs because it didn't quite manipulate things in the way they expected. This can be especially problematic when people have more complex use-cases.

If you've tried doing more complex things with policies you may have encountered examples where Amazon just disable the "simple" UI because your policy can't be represented that way

Comment on lines +139 to +140
module.warn(json.dumps(updated_acl))

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like you've left some debugging behind.

Comment on lines +184 to +190
if target_access == "acl":
for p in current_policy.get("Statement", []):
if p != policy_statement:
new_policy_statements.append(p)
new_policy["Statement"] = new_policy_statements
if new_policy == current_policy:
return False
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the policy being modified if the access type is acl? It's not like you go back and clean up the policy when logging is disabled.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point that I have missed. I think that these leftovers are harmless and I don't know what is the common practice, but I believe it should cleanup. Should I add a cleanup for when the logging is disabled, or remove it this part

@abraverm
Copy link
Author

@tremble thank you for the feedback, and I have created #2136 as alternative to this PR.
But maybe it would be better to extend ACL management in s3_bucket module, and remove it for s3_logging?

@tremble
Copy link
Contributor

tremble commented Aug 31, 2024

But maybe it would be better to extend ACL management in s3_bucket module, and remove it for s3_logging?

Yeah, I think long term it is better handled in s3_bucket.

If we're removing the grant management from s3_logging we'll need to go through a deprecation period anyway since it'll be a breaking change.

@abraverm
Copy link
Author

abraverm commented Sep 4, 2024

Closing in favor of #2136

@abraverm abraverm closed this Sep 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants