Overwriting tags when merging documents

[Sleavely]

I've got a main configuration file that I want to merge some overrides into, so I followed the example "Merge / combine all documents into one" which outlines the following approach:

yq ea -i '. as $item ireduce ({}; . * $item )' file1.yml file2.yml

And it works well as long as file2.yml doesnt attempt to modify any entries that use tags like !Sub which is a common shorthand in CloudFormation syntax.

file1.yml

Globals:
  Function:
    Environment:
      Variables:
        API_BASE_PATH: !Sub ${PROJECT}

file2.yml

Globals:
  Function:
    Environment:
      Variables:
        API_BASE_PATH: !If
          - IsProd
          - ""
          - !Sub ${PROJECT}

The result is an interesting hybrid:

Globals:
  Function:
    Environment:
      Variables:
        API_BASE_PATH: !Sub
          - IsProd
          - ""
          - !Sub ${PROJECT}

The list of arguments for the !If is appropriately injected but the tag itself doesnt seem to overwrite !Sub. Does anyone have any ideas on whether its possible to treat the tags as scalar values or some similar approach?

[Sleavely]

I tried skipping the tag entirely in file2.yml to use the longform Fn::If which should be interpreted as a regular map:

Globals:
  Function:
    Environment:
      Variables:
        API_BASE_PATH:
          Fn::If:
          - IsProd
          - ""
          - !Sub ${PROJECT}

but the !Sub tag remains after API_BASE_PATH:

Globals:
  Function:
    Environment:
      Variables:
        API_BASE_PATH: !Sub
          Fn::If:
            - IsProd
            - ""
            - !Sub ${PROJECT}

[mikefarah]

Yeah I see - I actually put in specific logic to keep the tags (as part of an issue raised a while back) see https://mikefarah.gitbook.io/yq/operators/multiply-merge#custom-types-that-are-really-maps

Seems like I need another multiplier flag to allow overriding of tags...

[Sleavely]

Oh 😅 I take it its not possible to work around with the existing operators?

For my specific case I can live with doing a stringreplace on !Sub tags that exist at EOL after merging the files but I think it'd be a useful feature for similar cases.

Regardless, thanks for the tool 🙏

[mikefarah]

Hmm the only work around I can think of is to explicitly set the tag after the fact. But then you would need knowledge of which node to set the tag on, e.g.:

yq ea '. as $item ireduce ({}; . * $item ) | .Globals.Function.Environment.Variables.API_BASE_PATH tag = ""' file1.yaml file2.yaml

[Sleavely]

Yeah that seems a little rough around the edge when trying to just "override with whatever is in file2.yaml". I guess I'll have to live with the sed -i 's/!Sub$/!If/' approach for now. Thanks :)

[mikefarah]

The latest 4.27.3 release has a new merge flag 'c' to 'clobber' custom tags - give it a go

yq ea -i '. as $item ireduce ({}; . *c $item )' file1.yml file2.yml

[Sleavely]

Haven't had time to test it out until now but it works like a charm functionally. I'm seeing some warnings related to guessTagFromCustomType being printed but might be unrelated - the output itself looks just as I want it to 👌