Skip to content

Latest commit

 

History

History
213 lines (165 loc) · 7.5 KB

README.md

File metadata and controls

213 lines (165 loc) · 7.5 KB
Raw

YAML interpolate

yaml_interpolate is a command-line tool for doing string interpolation inside a YAML file.

That tool exists, since YAML specification not allow string interploation, only alieses and anchors.

That tool is designed to process a single YAML file with both template and data for it. See possible alternatives (1, 2, 3, 4) for other cases / other languages.

Written in C++11 (not in python or ruby, for example), since I find it hard to express logic without strong typing :)

Command-line options

Usage:
  ./yaml_interpolate [OPTION...]

 I/O options:
  -i, --input PATH   Input yaml file path (or 'stdin'). (required)
  -o, --output PATH  Output file name (or 'stdout') (default: stdout)

 Parsing settings options:
      --formats LIST      Formats:
                          moustache:      {{ variable.path }}
                          double_dollar:  $variable.path$
                          double_percent: %variable.path%
                           (default: moustache)
      --separator STRING  Separator between node names (default: .)
      --regexps LIST      Regular expressions to find pattern
                          Notice, that them must contain single capture group

 Help options:
      --help  Print this help

Example of usage

$ yaml_interpolate --input=input.yaml --output=processed.yaml --formats=moustache
strings:
    str1: "Hello"
    str2: "world"

# Interpolated value: "Hello, world!"
greeting: "{{strings.str1}}, {{strings.str2}}!"

Advanced usage

Access by index

numbers:
    - 2.71828
    - 6.022e+23
    - value: 3.14

# Interpolated value: "Numbers: [ 2.71828, 6.022e+23, 3.14 ]"
text: "Numbers: [ {{numbers.0}}, {{numbers.1}}, {{numbers.2.value}} ]"

Multiple formats support:

  • Inline form (single formats option): 
    $ yaml_interpolate --input=input.yaml --output=processed.yaml \
    --formats=moustache,double_dollar,double_percent
  • Multi-option form (multiple formats options): 
    $ yaml_interpolate --input=input.yaml --output=processed.yaml \
    --formats=moustache \
    --formats=double_dollar \
    --formats=double_percent
strings:
    str1: "Hello"
    str2: "world"

# Interpolated value: "Hello, world!"
greeting1: "{{ strings.str1 }}, {{ strings.str2 }}!" # <-- format: moustache
greeting2: "$strings.str1$, $strings.str2$!"         # <-- format: double_dollar
greeting3: "%strings.str1%, %strings.str2%!"         # <-- format: double_percent

# Interpolated value: "Greetings: Hello, world!, Hello, world!, Hello, world!"
greetings: "Gretings: %greeting1%, $greeting2$, {{greeting3}}" # <-- Composite

Adding custom regular expressions

For example, here is used the next simple regular expressions:

  • \%\{(\S+)\} - parse "%{tokens}" pattern
  • \$\{(\S+)\} - parse "${tokens}" pattern
  • \$\[(\S+)\] - parse "$[tokens]" pattern

Notice, that some characters in regular expressions must be escaped twice!

For example - passing \${\{(\S+)\} option will be interpreted as $\{(\S+)\} (not \$ at start, but $ - not what we want).

It must be rewritten in the next forms:

  • \\${\{(\S+)\} - escaping only \$ at start
  • \\$\\{(\\S+)\\} - escaping all special characters (safe way, preferred)
  • [\$][\{](\S+)[\}] - non-escaping way, wrapping special characters. Interpreted as: [$][\{](\S+)[\}]

Read about escaping characters in shell here.

  • Inline form (single regexps option): 
    $ yaml_interpolate --input=input.yaml --output=processed.yaml \
    --formats=moustache \
    --regexps="\\%\\{(\\S+)\\}","\\$\\{(\\S+)\\}","\\$\\[(\\S+)\\]"
  • Multi-option form (multiple regexps options): 
    $ yaml_interpolate --input=input.yaml --output=processed.yaml \
    --formats=moustache \
    --regexps="\\%\\{(\\S+)\\}" \
    --regexps="\\$\\{(\\S+)\\}" \
    --regexps="\\$\\[(\\S+)\\]"
constants:
    pi: 3.1415
    e:  2.7182

# Interpolated value: "[ 3.1415, 2.7182, 3.1415, 2.7182 ]"
text: "[ {{constants.pi}}, %{constants.e}, ${constants.pi}, $[constants.e] ]"

Multiple documents processing

Multi-document processing supported - a YAML character stream may contain several documents. Each document is completely independent from the rest.

---
name:  "Tomato"
color: "red"
text: "{{name}} is {{color}}" # <-- Interpolated value: "Tomato is red"
---
name: "Kiwi"
color: "green"
text: "{{name}} is {{color}}" # <-- Interpolated value: "Kiwi is green"
---
name:  "Blueberry"
color: "blue"
text: "{{name}} is {{color}}" # <-- Interpolated value: "Blueberry is blue"

Piping

Piping is possible by redirecting output into stdout - for reading by another tool. For example - redirecting into yq (analogue of jq):

$ yaml_interpolate --input=input.yaml --output=stdout --formats=moustache | yq eval '.some.field' -

Reading from stdin is also possible - to use this tool in a pipeline. For example:

  1. Passing minified YAML string
  2. Processing by yaml_interpolate
  3. Extraction from text node: "Hello, world!"
$ echo "{data: {str: world}, text: 'Hello, {{data.str}}!'}" | yaml_interpolate --input=stdin --output=stdout | yq eval '.text' -

Limitations

  • You must always specify full node path - relative search not allowed, since the content of a YAML file constitutes a directed graph, not a tree. 
    root_node:
    leaf1: "Leaf 1 text"
    leaf2: "Leaf 2 text"
    leaf3: "{{leaf1}} & {{leaf2}}" # <-- Dont works, use '{{root_node.leaf1}} & {{root_node.leaf2}}'
    leaf4: "{{.leaf5.leaf1}} & {{.leaf5.leaf2}}" # <-- Dont works, use '{{root_node.leaf5.leaf1}} & {{root_node.leaf5.leaf2}}'
    leaf5:
        leaf1: "Leaf 6 text"
        leaf2: "Leaf 7 text"
  • Be careful and avoid circular depency and recursion. 
    root_node: # Dont do this:
    leaf1: "{{root_node.leaf2}}"
    leaf2: "{{root_node.leaf1}}"
    leaf3: "{{root_node.leaf2}} & {{root_node.leaf3}}"
  • Nodes keys not transformed, only values.

Dependencies

Build

$ mkdir build
$ cd build
$ bash ../build.sh ../