Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

amazon.aws.rds_cluster Parameter validation failed:\nUnknown parameter in input: \"BackupRetentionPeriod\", must be one of: DBClusterIdentifier\nUnknown parameter in input: \"ApplyImmediately\", must be one of: DBClusterIdentifier"} #2197

Closed
1 task done
fk-brambles opened this issue Jul 17, 2024 · 11 comments · Fixed by #2215
Labels

Comments

@fk-brambles
Copy link

Summary

when trying to interact with an RDS cluster (using amazon.aws.rds_cluster), it fails with

Parameter validation failed:\nUnknown parameter in input: "BackupRetentionPeriod", must be one of: DBClusterIdentifier\nUnknown parameter in input: "ApplyImmediately", must be one of: DBClusterIdentifier"}

Issue Type

Bug Report

Component Name

amazon.aws.rds_cluster

Ansible Version

ansible [core 2.17.2]
config file = /etc/ansible/ansible.cfg
configured module search path = ['/home/myuser/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/myuser/ansible-10.1.0/lib/python3.11/site-packages/ansible
ansible collection location = /home/myuser/.ansible/collections:/usr/share/ansible/collections
executable location = /home/myuser/ansible-10.1.0/bin/ansible
python version = 3.11.6 (main, Nov 2 2023, 15:15:45) [GCC 8.5.0 20210514 (Red Hat 8.5.0-18)] (/home/myuser/ansible-10.1.0/bin/python3)
jinja version = 3.1.4
libyaml = True

Collection Versions

/home/myuser/.ansible/collections/ansible_collections

Collection Version


amazon.aws 8.1.0
ansible.posix 1.3.0
ansible.windows 2.1.0
community.aws 6.4.0
community.general 4.0.0
community.libvirt 1.2.0
community.postgresql 2.1.5
containers.podman 1.9.3

/home/myuser/ansible-10.1.0/lib/python3.11/site-packages/ansible_collections

Collection Version


amazon.aws 8.0.1
ansible.netcommon 6.1.3
ansible.posix 1.5.4
ansible.utils 4.1.0
ansible.windows 2.4.0
arista.eos 9.0.0
awx.awx 24.5.0
azure.azcollection 2.4.0
check_point.mgmt 5.2.3
chocolatey.chocolatey 1.5.1
cisco.aci 2.9.0
cisco.asa 5.0.1
cisco.dnac 6.16.0
cisco.intersight 2.0.9
cisco.ios 8.0.0
cisco.iosxr 9.0.0
cisco.ise 2.9.2
cisco.meraki 2.18.1
cisco.mso 2.6.0
cisco.nxos 8.1.0
cisco.ucs 1.10.0
cloud.common 3.0.0
cloudscale_ch.cloud 2.3.1
community.aws 8.0.0
community.ciscosmb 1.0.9
community.crypto 2.20.0
community.digitalocean 1.26.0
community.dns 3.0.1
community.docker 3.10.4
community.general 9.1.0
community.grafana 1.9.1
community.hashi_vault 6.2.0
community.hrobot 2.0.1
community.library_inventory_filtering_v1 1.0.1
community.libvirt 1.3.0
community.mongodb 1.7.4
community.mysql 3.9.0
community.network 5.0.3
community.okd 3.0.1
community.postgresql 3.4.1
community.proxysql 1.5.1
community.rabbitmq 1.3.0
community.routeros 2.16.0
community.sap_libs 1.4.2
community.sops 1.6.7
community.vmware 4.4.0
community.windows 2.2.0
community.zabbix 2.5.1
containers.podman 1.15.2
cyberark.conjur 1.3.0
cyberark.pas 1.0.25
dellemc.enterprise_sonic 2.4.0
dellemc.openmanage 9.3.0
dellemc.powerflex 2.5.0
dellemc.unity 2.0.0
f5networks.f5_modules 1.28.0
fortinet.fortimanager 2.5.0
fortinet.fortios 2.3.6
frr.frr 2.0.2
google.cloud 1.3.0
grafana.grafana 5.2.0
hetzner.hcloud 3.1.1
ibm.qradar 3.0.0
ibm.spectrum_virtualize 2.0.0
ibm.storage_virtualize 2.3.1
ieisystem.inmanage 2.0.0
infinidat.infinibox 1.4.5
infoblox.nios_modules 1.6.1
inspur.ispim 2.2.3
inspur.sm 2.3.0
junipernetworks.junos 8.0.0
kaytus.ksmanage 1.2.2
kubernetes.core 3.2.0
lowlydba.sqlserver 2.3.3
microsoft.ad 1.6.0
netapp.cloudmanager 21.22.1
netapp.ontap 22.11.0
netapp.storagegrid 21.12.0
netapp_eseries.santricity 1.4.0
netbox.netbox 3.19.1
ngine_io.cloudstack 2.3.0
ngine_io.exoscale 1.1.0
openstack.cloud 2.2.0
openvswitch.openvswitch 2.1.1
ovirt.ovirt 3.2.0
purestorage.flasharray 1.28.1
purestorage.flashblade 1.17.0
sensu.sensu_go 1.14.0
splunk.es 3.0.0
t_systems_mms.icinga_director 2.0.1
telekom_mms.icinga_director 2.1.2
theforeman.foreman 4.0.0
vmware.vmware_rest 3.0.1
vultr.cloud 1.13.0
vyos.vyos 4.1.0
wti.remote 1.0.5

AWS SDK versions

Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: [email protected]
License: MIT
Location: /home/myuser/ansible-10.1.0/lib/python3.11/site-packages
Requires:
Required-by:

Name: boto3
Version: 1.34.144
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /home/myuser/ansible-10.1.0/lib/python3.11/site-packages
Requires: botocore, jmespath, s3transfer
Required-by:

Name: botocore
Version: 1.34.144
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /home/myuser/ansible-10.1.0/lib/python3.11/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer

Configuration

CONFIG_FILE() = /etc/ansible/ansible.cfg
DEFAULT_FORKS(/etc/ansible/ansible.cfg) = 50
DEFAULT_VAULT_PASSWORD_FILE(/etc/ansible/ansible.cfg) = /home/myuser/.vaultPassword
INTERPRETER_PYTHON(/etc/ansible/ansible.cfg) = auto_silent
PERSISTENT_COMMAND_TIMEOUT(/etc/ansible/ansible.cfg) = 10
PERSISTENT_CONNECT_RETRY_TIMEOUT(/etc/ansible/ansible.cfg) = 30
PERSISTENT_CONNECT_TIMEOUT(/etc/ansible/ansible.cfg) = 30

OS / Environment

RHEL8

Steps to Reproduce

  • hosts: '{{ host }}'
    gather_facts: no
    ignore_unreachable: true
    tasks:
    • name : Stop database
      block:
      • name: "Assume role"
        community.aws.sts_assume_role:
        role_arn: "{{ aws_role_to_assume }}"
        role_session_name: test-rds
        register : assumed_role
        delegate_to: localhost
      • name: "Stop the DB {{ aws_rds_cluster_id }}"
        amazon.aws.rds_cluster:
        aws_access_key: "{{ assumed_role.sts_creds.access_key }}"
        aws_secret_key: "{{ assumed_role.sts_creds.secret_key }}"
        session_token: "{{ assumed_role.sts_creds.session_token }}"
        db_cluster_identifier: "{{ aws_rds_cluster_id }}"
        region: "{{ aws_region }}"
        state: stopped
        apply_immediately: true
        delegate_to: localhost
        tags: ['stopdb', 'restartdb']

Expected Results

i expect the database to stop

Actual Results

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: Unknown parameter in input: "ApplyImmediately", must be one of: DBClusterIdentifier
[WARNING]: Module did not set no_log for force_update_password
fatal: [rds-mydb-tst -> localhost]: FAILED! => {"boto3_version": "1.34.144", "botocore_version": "1.34.144", "changed": false, "msg": "Unexpected failure for method stop_db_cluster with parameters {'BackupRetentionPeriod': 1, 'DBClusterIdentifier': 'aurpostdb-mydb-devtest1', 'ApplyImmediately': True}: Parameter validation failed:\nUnknown parameter in input: "BackupRetentionPeriod", must be one of: DBClusterIdentifier\nUnknown parameter in input: "ApplyImmediately", must be one of: DBClusterIdentifier"}

Code of Conduct

  • I agree to follow the Ansible Code of Conduct
@GomathiselviS GomathiselviS added needs_verified Some one might want to take a look at this and reproduce it to confirm and removed needs_triage labels Jul 23, 2024
@hakbailey
Copy link
Contributor

@fk-brambles Thank you for submitting this issue! Would you be willing to submit a PR to fix it?

@hakbailey hakbailey added jira and removed needs_verified Some one might want to take a look at this and reproduce it to confirm labels Jul 24, 2024
@fk-brambles
Copy link
Author

fk-brambles commented Aug 8, 2024 via email

@mandar242
Copy link
Contributor

Hello Helen thanks for contacting me. I am not familiar with Pull Requests. what would you like me to do? thanks for your help and best regards Francois

________________________________ From: Helen Bailey @.> Sent: Wednesday, July 24, 2024 8:34 AM To: ansible-collections/amazon.aws @.> Cc: Francois Keen @.>; Mention @.> Subject: Re: [ansible-collections/amazon.aws] amazon.aws.rds_cluster Parameter validation failed:\nUnknown parameter in input: "BackupRetentionPeriod", must be one of: DBClusterIdentifier\nUnknown parameter in input: "ApplyImmediately", must be one of: DBCl... @ fk-brambles Thank you for submitting this issue! Would you be willing to submit a PR to fix it? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned. Message ID: <ansible-collections/amazon. aws/issues/2197/2247801458@ github. com> ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd @fk-brambleshttps://urldefense.com/v3/__https://github.com/fk-brambles__;!!HDu-1JFExg!U3ZHS7mcVJ05HfD2LwjBa8Wa-vkp-zKBENLlHlRXY6ldvMxbJeZ74yolvOr6lbVbg3KWpxxEq4IPRxE8xdjkDbyB0WsT$ Thank you for submitting this issue! Would you be willing to submit a PR to fix it? — Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/ansible-collections/amazon.aws/issues/2197*issuecomment-2247801458__;Iw!!HDu-1JFExg!U3ZHS7mcVJ05HfD2LwjBa8Wa-vkp-zKBENLlHlRXY6ldvMxbJeZ74yolvOr6lbVbg3KWpxxEq4IPRxE8xdjkDT-OWVrX$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/BFJX5E64SKVOUZHJNXEZHFLZN6NNDAVCNFSM6AAAAABLA7QRTKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBXHAYDCNBVHA__;!!HDu-1JFExg!U3ZHS7mcVJ05HfD2LwjBa8Wa-vkp-zKBENLlHlRXY6ldvMxbJeZ74yolvOr6lbVbg3KWpxxEq4IPRxE8xdjkDStFMuJy$. You are receiving this because you were mentioned.Message ID: @.***> Visit our website: http://www.chep.com AVERTISSEMENT : ce courrier électronique ainsi que toutes les pièces jointes s’y rapportant doivent être exclusivement utilisés par les personnes auxquelles ils sont destinés. Ils sont susceptibles de contenir des informations confidentielles. Si vous n'êtes pas ni la personne destinataire de ce message, ni un salarié ou un mandataire chargé de transmettre ce message à son destinataire, veuillez noter que toute diffusion, distribution, copie ou autre utilisation du présent message et/ ou de ses pièces jointes est formellement interdite. Si vous avez reçu ce message par erreur, nous vous remercions d'en informer l'expéditeur immédiatement en répondant à son message et de le supprimer de votre ordinateur. Merci. Confidentiality Notice: This email is intended only for the use of the party to which it is addressed and may contain information that is privileged, confidential, or protected by law. If you are not the intended recipient you are hereby notified that any dissemination, copying or distribution of this email or its contents is strictly prohibited. If you have received this message in error, please notify us immediately by replying to the message and deleting it from your computer. .

Hi @fk-brambles , PR #2215 should solve the issue. Would be great if you could give it a try. Thanks!

@fk-brambles
Copy link
Author

fk-brambles commented Aug 8, 2024 via email

@mandar242
Copy link
Contributor

@fk-brambles based on my testing likely the PR changes should fix the issue as the code removes other parameters like BackupRetentionPeriod that are not expected by the method stop_db_clsuter.
Can you please check again if the playbook is using the correct code? also would be helpful if you can share the playbook that you are running, I could give it a try to reproduce the issue on my end. Thanks!

@fk-brambles
Copy link
Author

fk-brambles commented Aug 8, 2024 via email

patchback bot pushed a commit that referenced this issue Aug 27, 2024
SUMMARY

Fixes #2197
Limit params sent to api call to DBClusterIdentifier when using state started or stopped as
start_db_cluster and start_db_cluster supports only DBClusterIdentifier
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/rds/client/start_db_cluster.html
https://boto3.amazonaws.com/v1/documentation/api/1.26.120/reference/services/rds/client/stop_db_cluster.html

ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME

rds_cluster
ADDITIONAL INFORMATION

Reviewed-by: Alina Buzachis
Reviewed-by: Mark Chappell
(cherry picked from commit 7e8aad3)
@fk-brambles
Copy link
Author

thank you @mandar242 for the code update. I updated rds_cluster.py with the latest code and it is still not working, it still produces a:

[WARNING]: Module did not set no_log for force_update_password
fatal: [rds-APP0002662-tst -> localhost]: FAILED! => {"boto3_version": "1.34.144", "botocore_version": "1.34.144", "changed": false, "msg": "Unexpected failure for method stop_db_cluster with parameters {'BackupRetentionPeriod': 1, 'DBClusterIdentifier': 'aurpostdb-mydb-devtest1', 'ApplyImmediately': True}: Parameter validation failed:\nUnknown parameter in input: "BackupRetentionPeriod", must be one of: DBClusterIdentifier\nUnknown parameter in input: "ApplyImmediately", must be one of: DBClusterIdentifier"}

[WARNING]: Module did not set no_log for force_update_password
fatal: [rds-APP0002662-tst -> localhost]: FAILED! => {"boto3_version": "1.34.144", "botocore_version": "1.34.144", "changed": false, "msg": "Unexpected failure for method start_db_cluster with parameters {'BackupRetentionPeriod': 1, 'DBClusterIdentifier': 'aurpostdb-mydb-devtest1', 'ApplyImmediately': True}: Parameter validation failed:\nUnknown parameter in input: "BackupRetentionPeriod", must be one of: DBClusterIdentifier\nUnknown parameter in input: "ApplyImmediately", must be one of: DBClusterIdentifier"}

May you please have a look?
thank you

softwarefactory-project-zuul bot pushed a commit that referenced this issue Aug 28, 2024
This is a backport of PR #2215 as merged into main (7e8aad3).
SUMMARY

Fixes #2197
Limit params sent to api call to DBClusterIdentifier when using state started or stopped as
start_db_cluster and start_db_cluster supports only DBClusterIdentifier
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/rds/client/start_db_cluster.html
https://boto3.amazonaws.com/v1/documentation/api/1.26.120/reference/services/rds/client/stop_db_cluster.html

ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME

rds_cluster
ADDITIONAL INFORMATION

Reviewed-by: Mark Chappell
braydencw1 pushed a commit to braydencw1/amazon.aws that referenced this issue Aug 29, 2024
SUMMARY

Fixes ansible-collections#2197
Limit params sent to api call to DBClusterIdentifier when using state started or stopped as
start_db_cluster and start_db_cluster supports only DBClusterIdentifier
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/rds/client/start_db_cluster.html
https://boto3.amazonaws.com/v1/documentation/api/1.26.120/reference/services/rds/client/stop_db_cluster.html

ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME

rds_cluster
ADDITIONAL INFORMATION

Reviewed-by: Alina Buzachis
Reviewed-by: Mark Chappell
@randoneering
Copy link

randoneering commented Aug 29, 2024

Ran into the same issue as reported-I saw the code was merged to main to resolve the issue. But I assume this fix will not be applied until a new release? If so, where could I track this release? I didn't see a way to do so. Wrote a playbook to initiate a "lights out/ on" policy that shuts down rds_clusters and instances. Instance module seems to be fine, but the rds_cluster module is where this fails with the same error as the reporter.

  - block:
      - name: shutdown clusters
        amazon.aws.rds_cluster:
          region: us-east-1
          access_key: "{{ assumed_role_prod.sts_creds.access_key }}"
          secret_key: "{{ assumed_role_prod.sts_creds.secret_key }}"
          session_token: "{{ assumed_role_prod.sts_creds.session_token }}"
          db_cluster_identifier: '{{ item }}'
          state: stopped
          wait: false
        with_items: "{{ prod_clusters_nonexempt }}"
        when: prod_clusters_nonexempt is defined
        register: prod_clusters_shutdown
      - name: shutdown instances
        amazon.aws.rds_instance:
          region: us-east-1
          access_key: "{{ assumed_role_prod.sts_creds.access_key }}"
          secret_key: "{{ assumed_role_prod.sts_creds.secret_key }}"
          session_token: "{{ assumed_role_prod.sts_creds.session_token }}"
          db_instance_identifier: '{{ item }}'
          state: stopped
          wait: false
        with_items: "{{ prod_instances_nonexempt }}"
        when: prod_instances_nonexempt is defined
        register: prod_instances_shutdown
      rescue:
      - name: send notification via teams when shutdown fails
        community.general.office_365_connector_card:
          webhook: "{{ webhook }}"
          title: "LightsOut playbook failed for Prod"
          summary: "The LightsOut playbook experienced an error "
          text: "Check job {{ tower_job_id }} for LightsOut playbook failure"

@tremble
Copy link
Contributor

tremble commented Sep 2, 2024

@randoneering

The change has also been backported to our "stable-8" branch (#2252) which means that it should be available once 8.2.0 is available

#2277 is one of the last housekeeping pieces prior to the release of 8.2.0 so the best place to watch is there. Assuming there aren't any last minute surprises, I hope 8.2.0 will be available within the next 24 hours...

For what it's worth, we try to follow a roughly monthly release cadence for minor releases, and roughly a 6-month release cadence for major releases (usually bumping at least the botocore requirements). In theory you can watch this repo for "releases" (which should be triggered by the same process that performs the release into Ansible Galaxy), but it's not something I've tried before...

@fk-brambles
Copy link
Author

thank you all and @mandar242 and @tremble - I confirm that with amazon.aws collection 8.2.0, the stop start works. thank you.

I noticed that with the parameter "state: stopped", it doesn't wait for the DB status be fully "Stopped" (i.e. the Ansible module returns "changed" pretty much instantly as soon as the DB status is "stopping"), whereas with the parameter "state: started", it waits for DB status to be "available". Should I log this as an issue or is it indented to behave like this?

Thanks again for all your support.

@randoneering
Copy link

@tremble thank you for the info! I too, can confirm, that the fix is working for the rds_cluster module now. Thank you all for the help/info.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants