yaml.composer.ComposerError: found duplicate anchor 'everything'; first occurrence

[RafalSkolasinski]

SUMMARY

I have an role that automates installation of Knative Eventing & Serving. It does work fine until the 0.22.0 version of knative but fails for 0.24.0 (maybe 0.23.0 too as well but didn't try).

Using kubectl instead still works fine.

ISSUE TYPE

• Bug Report

COMPONENT NAME

community.kubernetes.k8s

ANSIBLE VERSION

ansible [core 2.11.2] 
  config file = None
  configured module search path = ['/home/rskolasinski/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/rskolasinski/.asdf/installs/python/3.8.5/lib/python3.8/site-packages/ansible
  ansible collection location = /home/rskolasinski/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/rskolasinski/.asdf/installs/python/3.8.5/bin/ansible
  python version = 3.8.5 (default, Jun 23 2021, 15:05:48) [GCC 9.3.0]
  jinja version = 2.11.3
  libyaml = True

COLLECTION VERSION

# /home/rskolasinski/.ansible/collections/ansible_collections
Collection           Version  
-------------------- ---------
community.crypto     1.7.1    
community.docker     1.8.0    
community.kubernetes 1.2.1    
kubernetes.core      2.1.1    
seldonio.k8s         0.2.0-dev

CONFIGURATION

ANSIBLE_NOCOWS(/home/rskolasinski/work/ansible-k8s-collection/ansible.cfg) = True
DEFAULT_HOST_LIST(/home/rskolasinski/work/ansible-k8s-collection/ansible.cfg) = ['/home/rskolasinski/work/ansible-k8s-collection/inventory.in>
DEFAULT_ROLES_PATH(/home/rskolasinski/work/ansible-k8s-collection/ansible.cfg) = ['/home/rskolasinski/work/ansible-k8s-collection/roles']

OS / ENVIRONMENT

Ubuntu 20.04 against k8s v1.19.9-gke.1900

STEPS TO REPRODUCE

Use following task

- name: Install Knative Eventing
  community.kubernetes.k8s:
    state: present
    definition: "{{ lookup('url', item, split_lines=False) }}"
  with_items:
    - "{{ knative_eventing_crds }}"
    - "{{ knative_eventing_core }}"
    - "{{ knative_eventing_channel }}"
    - "{{ knative_eventing_broker }}"

with these variables

knative_eventing_crds: "https://github.com/knative/eventing/releases/download/v0.24.0/eventing-crds.yaml"
knative_eventing_core: "https://github.com/knative/eventing/releases/download/v0.24.0/eventing-core.yaml"
knative_eventing_channel: "https://github.com/knative/eventing/releases/download/v0.24.0/in-memory-channel.yaml"
knative_eventing_broker: "https://github.com/knative/eventing/releases/download/v0.24.0/mt-channel-broker.yaml"

EXPECTED RESULTS

Installation successful./

ACTUAL RESULTS

Following error (formatted for better view) is reported by Ansible

{"ansible_loop_var": "item", "changed": false, "item": "https://github.com/knative/eventing/releases/download/v0.24.0/in-memory-channel.yaml", "module_stderr": "Traceback (most recent call last):
  File "/home/rskolasinski/.ansible/tmp/ansible-tmp-1627477047.1584105-1539525-132404581122003/AnsiballZ_k8s.py", line 100, in <module>
    _ansiballz_main()
  File "/home/rskolasinski/.ansible/tmp/ansible-tmp-1627477047.1584105-1539525-132404581122003/AnsiballZ_k8s.py", line 92, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/home/rskolasinski/.ansible/tmp/ansible-tmp-1627477047.1584105-1539525-132404581122003/AnsiballZ_k8s.py", line 40, in invoke_module
    runpy.run_module(mod_name='ansible_collections.kubernetes.core.plugins.modules.k8s', init_globals=dict(_module_fqn='ansible_collections.kubernetes.core.plugins.modules.k8s', _modlib_path=modlib_path),
  File "/home/rskolasinski/.asdf/installs/python/3.8.5/lib/python3.8/runpy.py", line 207, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/home/rskolasinski/.asdf/installs/python/3.8.5/lib/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/home/rskolasinski/.asdf/installs/python/3.8.5/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_kubernetes.core.k8s_payload_fmbp5s8o/ansible_kubernetes.core.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/modules/k8s.py", line 380, in <module>
  File "/tmp/ansible_kubernetes.core.k8s_payload_fmbp5s8o/ansible_kubernetes.core.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/modules/k8s.py", line 376, in main
  File "/tmp/ansible_kubernetes.core.k8s_payload_fmbp5s8o/ansible_kubernetes.core.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/modules/k8s.py", line 359, in execute_module
  File "/tmp/ansible_kubernetes.core.k8s_payload_fmbp5s8o/ansible_kubernetes.core.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/common.py", line 465, in set_resource_definitions
  File "/tmp/ansible_kubernetes.core.k8s_payload_fmbp5s8o/ansible_kubernetes.core.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/common.py", line 465, in <listcomp>
  File "/home/rskolasinski/.asdf/installs/python/3.8.5/lib/python3.8/site-packages/yaml/__init__.py", line 130, in load_all
    yield loader.get_data()
  File "/home/rskolasinski/.asdf/installs/python/3.8.5/lib/python3.8/site-packages/yaml/constructor.py", line 45, in get_data
    return self.construct_document(self.get_node())
  File "/home/rskolasinski/.asdf/installs/python/3.8.5/lib/python3.8/site-packages/yaml/composer.py", line 27, in get_node
    return self.compose_document()
  File "/home/rskolasinski/.asdf/installs/python/3.8.5/lib/python3.8/site-packages/yaml/composer.py", line 55, in compose_document
    node = self.compose_node(None, None)
  File "/home/rskolasinski/.asdf/installs/python/3.8.5/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/home/rskolasinski/.asdf/installs/python/3.8.5/lib/python3.8/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/home/rskolasinski/.asdf/installs/python/3.8.5/lib/python3.8/site-packages/yaml/composer.py", line 82, in compose_node
    node = self.compose_sequence_node(anchor)
  File "/home/rskolasinski/.asdf/installs/python/3.8.5/lib/python3.8/site-packages/yaml/composer.py", line 111, in compose_sequence_node
    node.value.append(self.compose_node(node, index))
  File "/home/rskolasinski/.asdf/installs/python/3.8.5/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/home/rskolasinski/.asdf/installs/python/3.8.5/lib/python3.8/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/home/rskolasinski/.asdf/installs/python/3.8.5/lib/python3.8/site-packages/yaml/composer.py", line 75, in compose_node
    raise ComposerError("found duplicate anchor %r; first occurrence"
yaml.composer.ComposerError: found duplicate anchor 'everything'; first occurrence
  in "<unicode string>", line 862, column 12:
        verbs: &everything
               ^
second occurrence
  in "<unicode string>", line 921, column 12:
        verbs: &everything
               ^
", "module_stdout": "", "msg": "MODULE FAILURE
See stdout/stderr for the exact error", "rc": 1}

[gravesm]

@RafalSkolasinski this error is coming from pyyaml because there are duplicate anchors in the https://github.com/knative/eventing/releases/download/v0.24.0/in-memory-channel.yaml document. Unfortunately, there's not much we can do about this on our side. There is an open PR in the pyyaml repo for this. I'd guess the reason this works with kubectl is because the Go yaml parser it's using doesn't have this problem.

[Akasurde]

@RafalSkolasinski Could you please confirm if @gravesm's suggestion works for you? Thanks.

[gravesm]

I'm not sure there's any real suggestion here. The only workaround I can think of until pyyaml is fixed would be to just call out to kubectl. Not great, but the best I can offer. I'm going to go ahead and close this since it is a pyyaml problem. Feel free to reopen if you think there's some way we can address this.