step template withItems: template name '{{item}}' undefined

[aapjeisbaas]

Pre-requisites

• I have double-checked my configuration
• I have tested with the :latest image tag (i.e. quay.io/argoproj/workflow-controller:latest) and can confirm the issue still exists on :latest. If not, I have explained why, in detail, in my description below.
• I have searched existing issues and could not find a match for this bug
• I'd like to contribute the fix myself (see contributing guide)

What happened? What did you expect to happen?

I was hoping to call multiple templates by using withItems in my step

It errors with:

Failed: invalid spec: templates.make-magic.steps[0].k8s-create template name '{{item}}' undefined

If this would work I could make a compact deploy and cleanup of multiple k8s objects

The problem is in the steps:

    - - name: k8s-create
        template: "{{item}}"
        arguments:
          parameters:
          - name: action
            value: create
        withItems:
        - k8s-config-a
        - k8s-config-b
    - - name: work
        template: work
    - - name: k8s-delete
        template: "{{item}}"
        arguments:
          parameters:
          - name: action
            value: delete
        withItems:
        - k8s-config-a
        - k8s-config-b

You can write the steps in a more verbose way to make it just work without the withItems:

    - - name: k8s-create-a
        template: k8s-config-a
        arguments:
          parameters:
          - name: action
            value: create
      - name: k8s-create-b
        template: k8s-config-b
        arguments:
          parameters:
          - name: action
            value: create
    - - name: work
        template: work
    - - name: k8s-delete-a
        template: k8s-config-a
        arguments:
          parameters:
          - name: action
            value: delete
      - name: k8s-delete-b
        template: k8s-config-b
        arguments:
          parameters:
          - name: action
            value: delete

Version(s)

quay.io/argoproj/workflow-controller@sha256:f61c2eaf06294f38dc66f7519ae2c3af916ce7dcde8322999789bd39c29e983c

Paste a minimal workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: pretty-workflow-
  annotations:
    workflows.argoproj.io/description: |
      This workflow creates multiple k8s objects on start and cleans them up after the workflow.
spec:
  entrypoint: make-magic
  podGC:
    strategy: OnWorkflowSuccess
  templates:
  - name: make-magic
    steps:
    - - name: k8s-create
        template: "{{item}}"
        arguments:
          parameters:
          - name: action
            value: create
        withItems:
        - k8s-config-a
        - k8s-config-b
    - - name: work
        template: work
    - - name: k8s-delete
        template: "{{item}}"
        arguments:
          parameters:
          - name: action
            value: delete
        withItems:
        - k8s-config-a
        - k8s-config-b

  - name: k8s-config-a
    inputs:
      parameters:
      - name: action
    resource:
      action: "{{inputs.parameters.action}}"
      setOwnerReference: true
      manifest: |
        apiVersion: v1
        kind: ConfigMap
        metadata:
          name: awesome-config-a
          namespace: argo
        data:
          config.json: |-
            {
                "config_a": "true",
            }

  - name: k8s-config-b
    inputs:
      parameters:
      - name: action
    resource:
      action: "{{inputs.parameters.action}}"
      setOwnerReference: true
      manifest: |
        apiVersion: v1
        kind: ConfigMap
        metadata:
          name: awesome-config-b
          namespace: argo
        data:
          config.json: |-
            {
                "config_b": "true",
            }

  - name: work
    volumes:
    - name: config-a-vol
      configMap:
        name: awesome-config-a
    - name: config-b-vol
      configMap:
        name: awesome-config-b
    script:
      image: alpine:latest
      command: [sh]
      volumeMounts:
      - name: config-a-vol
        mountPath: "/tmp/a/config.json"
        subPath: "config.json"
        readOnly: true
      - name: config-b-vol
        mountPath: "/tmp/b/config.json"
        subPath: "config.json"
        readOnly: true
      source: |
        cat /tmp/a/config.json
        cat /tmp/b/config.json

Logs from the workflow controller

kubectl logs -n argo deploy/workflow-controller | grep ${workflow}
time="2024-08-30T09:20:39.359Z" level=info msg="Processing workflow" Phase= ResourceVersion=61248 namespace=argo workflow=pretty-workflow-l4r5f
time="2024-08-30T09:20:39.359Z" level=info msg="Updated phase  -> Failed" namespace=argo workflow=pretty-workflow-l4r5f
time="2024-08-30T09:20:39.359Z" level=info msg="Updated message  -> invalid spec: templates.make-magic.steps[0].k8s-create template name '{{item}}' undefined" namespace=argo workflow=pretty-workflow-l4r5f
time="2024-08-30T09:20:39.359Z" level=info msg="Marking workflow completed" namespace=argo workflow=pretty-workflow-l4r5f
time="2024-08-30T09:20:39.359Z" level=error msg="Unable to set ExecWorkflow" error="templates.make-magic.steps[0].k8s-create template name '{{item}}' undefined" namespace=argo workflow=pretty-workflow-l4r5f
time="2024-08-30T09:20:39.366Z" level=info msg="Workflow update successful" namespace=argo phase=Failed resourceVersion=61250 workflow=pretty-workflow-l4r5f

Logs from in your workflow's wait container

kubectl logs -n argo -c wait -l workflows.argoproj.io/workflow=${workflow},workflow.argoproj.io/phase!=Succeeded

`No resources found in argo namespace.`

[agilgur5]

If I'm not mistaken, I think template doesn't itself support being dynamically templated. #12842 added support for this in templateRef, which has a template field, but it may not work on this field. cc @shuangkun

[aapjeisbaas]

Thanks @agilgur5 I also gave templateRef a try this works but requires an external workflow template which I was trying to avoid to keep it as compact as possible.

[shuangkun]

I'll try to have a look.

[shuangkun]

I tried it, and it seems to be a bit tricky to implement.