Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(cli): handle multi-resource yaml in offline lint. Fixes #12137 #13531

Conversation

wang-wayne
Copy link
Contributor

@wang-wayne wang-wayne commented Aug 30, 2024

Fixes #12137
Fixes #10608

Motivation

If there are multiple templates in one yaml file. The argo lint --offline can only get the first template.
Because the yaml.Unmarshal function is used to parse a single YAML document.

Modifications

Read yaml file with common.ParseObjects in offline mode.

Verification

e2e test.
argo lint --offline test.yaml

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: hello-world-template-local-arg-1
spec:
  templates:
    - name: hello-world
      inputs:
        parameters:
          - name: msg
            value: "hello world"
      container:
        image: docker/whalesay
        command: [cowsay]
        args: ["{{inputs.parameters.msg}}"]
---
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: hello-world-template-local-arg-2
spec:
  templates:
    - name: hello-world
      inputs:
        parameters:
          - name: msg
            value: "hello world"
      container:
        image: docker/whalesay
        command: [cowsay]
        args: ["{{inputs.parameters.msg}}"]
---
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: hello-world-local-arg-
spec:
  entrypoint: whalesay
  templates:
    - name: whalesay
      steps:
        - - name: hello-world
            templateRef:
              name: hello-world-template-local-arg-2
              template: hello-world

Add unittest case.

@agilgur5 agilgur5 changed the title fix: Fixed argo lint offline issue on multi-documents yaml. Fixes #12137 fix(cli): handle multi-resource yaml in offline lint. Fixes #12137 Aug 30, 2024
@agilgur5 agilgur5 added the area/cli The `argo` CLI label Aug 30, 2024
@kayhern kayhern force-pushed the fix-argo-lint-offline-issue-on-multi-documents-yaml-file branch from 693db02 to 70c5e90 Compare August 30, 2024 13:46
@agilgur5
Copy link

@miltalex do you want to try reviewing this? Since you've worked on argo lint before

@agilgur5 agilgur5 requested a review from miltalex August 31, 2024 23:21
@agilgur5
Copy link

I'm curious if this also fixes #10608?

@miltalex
Copy link
Member

miltalex commented Sep 1, 2024

@miltalex do you want to try reviewing this? Since you've worked on argo lint before

I would be happy to review it

@wang-wayne
Copy link
Contributor Author

wang-wayne commented Sep 6, 2024

I'm curious if this also fixes #10608?

@agilgur5 Yes, I had a validation. This PR also fixes #10608.

@wang-wayne
Copy link
Contributor Author

@miltalex do you want to try reviewing this? Since you've worked on argo lint before

I would be happy to review it

Hi @miltalex , thank you for reviewing my PR! Do you have any comments or suggestions to share? I would appreciate any feedback you might have. Thank you!

Copy link
Member

@miltalex miltalex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, thank you very much for submitting the fix. I have a small stylistic suggestion otherwise LGTM

cmd/argo/commands/lint_test.go Outdated Show resolved Hide resolved
cmd/argo/commands/lint_test.go Outdated Show resolved Hide resolved
@miltalex
Copy link
Member

/retest

@miltalex miltalex self-requested a review September 26, 2024 05:57
@wang-wayne wang-wayne requested a review from miltalex September 27, 2024 01:21
@kayhern kayhern force-pushed the fix-argo-lint-offline-issue-on-multi-documents-yaml-file branch from 899730b to 0c5b6aa Compare October 8, 2024 17:46
Signed-off-by: Wang, Wayne <[email protected]>
@kayhern kayhern force-pushed the fix-argo-lint-offline-issue-on-multi-documents-yaml-file branch from 651d412 to 3bb0c67 Compare October 9, 2024 18:51
Copy link

@agilgur5 agilgur5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work. This mostly LGTM, so I'll merge this, but there's some follow-ups that I think can be made in separate PRs to refactor this a tiny bit along with the existing code style it copies.

templateRef:
name: hello-world-template-local-arg-2
template: hello-world
`), 0644)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be rewritten for #13610, but that should probably be a separate PR for the whole file. Your current style matches the existing style of this file, so I think this is fine for now

cwftmpl := new(wfv1.ClusterWorkflowTemplate)
if err := yaml.Unmarshal(bytes, &cwftmpl); err != nil {
return fmt.Errorf("failed to unmarshal file %s as a ClusterWorkflowTemplate: %w", path, err)
for _, pr := range common.ParseObjects(bytes, false) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see this is copied from lint.go, so similarly I'd say this is fine for now, but a more clear name would be better in both places, like parsed instead of pr (which makes one think of "pull request")

Suggested change
for _, pr := range common.ParseObjects(bytes, false) {
for _, parsed := range common.ParseObjects(bytes, false) {

Comment on lines +58 to +59
if obj == nil {
continue // could not parse to kubernetes object

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This too is copied from lint.go, although I would think both places should have an error in this case rather than a silent continue.

namespace: wftmpl.Namespace,
workflowTemplates: map[string]*wfv1.WorkflowTemplate{},
objName := obj.GetName()
namespace := obj.GetNamespace()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here that this is copied from lint.go, but the naming could be consistent with objName as objNs in both places

Suggested change
namespace := obj.GetNamespace()
objNs := obj.GetNamespace()

@agilgur5 agilgur5 merged commit dfcaca4 into argoproj:main Oct 10, 2024
28 checks passed
@agilgur5 agilgur5 added this to the v3.5.x patches milestone Oct 10, 2024
isubasinghe pushed a commit that referenced this pull request Oct 30, 2024
isubasinghe pushed a commit that referenced this pull request Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/cli The `argo` CLI
Projects
None yet
3 participants