Skip to content

Latest commit

 

History

History
592 lines (495 loc) · 20.5 KB

_config-tool.adoc

File metadata and controls

592 lines (495 loc) · 20.5 KB

Configuration Tool

For the configuration of the .fed and .env files the Maven plugin uses Python scripts. The scripts can also be used as a standalone tool.

Build FED

The FED builder takes the .pol and .env archive and JSON files to configure the environmentalized fields and certificates of the project. The script is invoked with the jython interpreter provided with the package and deployment tools of the API Gateway.

$ buildfed.sh --help
Usage: buildfed.py OPTIONS

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -v, --verbose         Enable verbose messages [optional]
  -e FILEPATH, --env=FILEPATH
                        Path of environment archive (.env)
  -p FILEPATH, --pol=FILEPATH
                        Path of policy archive (.pol)
  -c FILEPATH, --config=FILEPATH
                        Path of JSON configuration file
  --prop=FILEPATH       Path of JSON property file [multiple]
  --cert=FILEPATH       Path of JSON certificate configuration file [optional]
  --cert-expiration=DAYS
                        Check if certificates expire within the next days
                        [optional]
  --cert-config-update  Enable writing of info section for 'update'
                        certificates within the configuration file [optional]
  --output-fed=FILEPATH
                        Path of output deployment archive file (.fed)
                        [optional]
  --output-env=FILEPATH
                        Path of output environment archive file (.env)
                        [optional]
  -D NAME:VALUE, --define=NAME:VALUE
                        Define a command line property [multiple]
  -F NAME:FILE, --fromFile=NAME:FILE
                        Define a command line property from file content [multiple]
  --passphrase-in=PASSPHRASE
                        Passphrase of input archive files [optional]
  --passphrase-out=PASSPHRASE
                        Passphrase for output archive files [optional]
  -s, --simulate        Enable simulation mode [optional]
  -b DIRECTORY, --base-dir=DIRECTORY
                        Base directory for certificate files [multiple]
  --secrets-file=FILEPATH
                        Path of JSON file containing confidential properties
                        [optional]
  --secrets-key=FILEPATH
                        Path to key file to decrypt confidential properties [optional]

If environmentalized fields or certificates are not configured, the build fails. Missing fields or certificates are automatically added to the configuration file.

Option Description

-e, --env

The .env file as generated by the projpack tool. The option is mandatory.

-p, --pol

The .pol file as generated by the projpack tool. The option is mandatory.

-c, --config

The JSON configuration file for environmentalized fields. The option is mandatory.

--prop

Path to a JSON file containing the property configuration. Multiple files are supported by specifying the parameter multiple times.

If a property occurs in multiple files the value from the last file is used.

Specify a single dash '-' as filename to read properties from stdin. The input must be a flat key/value JSON document (e.g., {"p1": "v1", "p2": "v2"}).

--cert

An optional JSON file containing the certificate configuration.

--cert-expiration

Optional number of days for certificate expiration check.

If at least one certificate expires with the next given days an error will be raised.

--cert-config-update

Enable writing of info section for 'update' certificates within the configuration file.

Since version v0.7.0 the info section of "update" certificates are not written any more. Use this to enable the previous behavior.

--output-fed

The path of the configured .fed file. If missing, no .fed file is generated.

--output-env

The path of the configured .env file. If missing, no .env file is generated.

--define

Define a command line property name:value. Command line properties can be used instead of custom defined properties. Multiple properties are supported by specifying the parameter multiple times.

--fromFile

Define a command line property from file content name:file. The file has to contain an UTF-8 encoded text. The content of the file will be placed into the defined property. Multiple files are supported by specifying the parameter multiple times.

--passphrase-in

Passphrase for source archives. This passphrase is used for reading the .pol and .env files.

For all source archives the same passphrase is used. Having different passphrases for .pol and .env file is not supported.

--passphrase-out

Passphrase for output archives. This passphrase is used on writing the configured archives. If not set the output archives have the same passphrase as the source archives.

-s, --simulation

Enable simulation mode.

In simulation mode, no output files (.fed or .env) will be written.

The simulation mode will ignore non existing certificate files and will not apply the values of environmentalized fields.

📎

The simulation mode can be used to prepare packages for CI/CD pipelines.

In this case configuration parameters and certificates may be stored in a configuration database. So during the preparation phase the certificates may not be available and the configuration files may contain placeholders. To avoid errors due to incompatible types (e.g. placeholder string used for an integer field) the values will not be applied to the entity store.

-b, --base-dir

Base directory for certificate files.

Multiple directories can be specified by adding the parameter multiple times.

If specified, the given directories are scanned for certificate files. If multiple certificate files exists the first one will be used.

--secrets-file

Path of JSON file containing confidential properties. The file has to be created by the encrypt tool.

--secrets-key

Path to key file to decrypt confidential properties.

The key file has to be the same as on creating the secrets file.

This parameter is required if a secrets file is specified.

Encrypt Secrets

The encrypt tools is used to generate an initial secrets file and to encrypt the values of the properties. The script is invoked with the jython interpreter provided with the package and deployment tools of the API Gateway.

The tool requires a path to the secrets file and a key file to encrypt the values. If the secrets file doesn’t exist a new file will be created. For existing files the given key is checked against the key used on file creation.

$ encrypt.sh -h
Usage: encrypt OPTIONS

Encrypt secrets.

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -v, --verbose         Enable verbose messages [optional]
  --secrets-file=FILEPATH
                        Path of JSON file containing confidential properties
  --secrets-key=FILEPATH
                        Path to key file to encrypt confidential properties
  --secrets-key-new=FILEPATH
                        Path to new key file to change key [optional]
Option Description

--secrets-file

Path of JSON file containing confidential properties.

--secrets-key

Path to key file to encrypt confidential properties.

Key File

The key file contains any arbitrary sequence of bytes. It is treated as a binary file.

If you edit this file with a text editor be aware of the encoding and the end of line sequence. In this case use ASCII characters in a single line (no line feed at the end) to prevent any incompatibility with other line end formats or editor encodings.

--secrets-key-new

Path to new key file.

Use this option to change the key. All values will be re-encrypted with the new key.

To add new properties tag the values with the encrypt: prefix. Values having this prefix will be encrypted on running the tool.

{
  "secrets": {
    "__": "3QjoMSfhSelmvMlvcgCdyHf+oTyVnHlyneA3stpN0iQKJ1BUIrY9OA==", (1)
    "my.password": "encrypt:changeme", (2)
    "cert.password": "eL5+ogfSxQue8+NA0/l859g/2nTFwxBUp/7l7z/sMOE=" (3)
  }
}

  1. Marker to check the key. Don’t delete or change it.

  2. The prefix encrypt: indicates that the value changeme has to be encrypted.

  3. Values without the prefix are already encrypted and will not be changed.

📎
 The encrypt tool use the same cipher as the entity store.

Examples

Encrypt values with the given key:

Command Line
$ encrypt.sh --secrets-file=gateway.crypt.json --secrets-key=secrets.key

Change the key of a secrets file:

Command Line
$ encrypt.sh --secrets-file=gateway.crypt.json --secrets-key=secrets.key --secrets-key-new=new_secrets.key

Configuration Files

For the configuration of the environment specific deployment archive, various configuration files are used. All configuration files are JSON documents. The schema of the JSON is not validated, so any arbitrary property can be added. Properties not known by the plugin will be ignored.

If a configuration file is updated by the plugin, all properties, including custom properties, are written.

💡
 When a configuration file is written by the plugin, all JSON properties are sorted. This makes diff & merge easy.

Following kind of configuration files are supported:

Table 1. Configuration Files
File Description Mandatory Updated [1]

Environmentalized fields

Configuration file for environmentalized fields.

yes

yes

Certificates

Configuration file for certificates stored in the project.

no

yes

Properties

Configuration file for properties references by the environmentalized fields and certificate configuration.

no

no

Secrets

Configuration file confidential properties.

no

no

Environmentalized Fields

For configuring environmentalized fields a JSON file having an entities property is used. For each configured environmentalized entity a property exists. The name of the property is the short hand key of the entity.

The value of the field can be specified directly by a value property. It can also be specified indirectly by a property or environment variables (controlled via the source property).

Properties are name/value pairs. The value of the property is defined …​ (in order of precedence)

  1. …​ as predefined system property.

  2. …​ as a command line property via the command line of the configuration tool.

  3. …​ within a separate property configuration file.

  4. …​ within the environmentalized fields file (under properties).

The plugin and the configuration tool provides a set of predefined system properties.

Table 2. Predefined System Properties
System Property Description Provided by

_system.artifact.group

Group of the project artifact.

Plugin

_system.artifact.name

Name of the project artifact.

Plugin

_system.artifact.ver

Version of the project artifact.

Plugin

_system.artifact.id

Complete ID of the project artifact.

Plugin

_system.artifact.info

JSON describing the artifact and its dependencies (content of the gateway.info.json file). If the gateway.info.json file is not available the property contains an empty JSON document {}.

Plugin

_system.build.datetime

Build date and time in ISO format (YYYY-MM-DD’T’HH:MM:SS.S)

Config Tool
src/main/axwgw/gateway.config.json
{
    "entities": { (1)
        "/[CircuitContainer]name=Hello World/[FilterCircuit]name=Hello World Message/[SetAttributeFilter]name=Set name": { (2)
            "description": "Name for the 'Hello World' message.",
            "fields": {
                "attributeValue#0": { (3)
                    "source": "property", (4)
                    "type": "string", (5)
                    "used": true, (6)
                    "value": "foobar" (7)
                }
            }
        },
        "/[CircuitContainer]name=Hello World/[FilterCircuit]name=Hello World Message/[SetAttributeFilter]name=Build time": {
            "description": "Build time in ISO format.",
            "fields": {
                "attributeValue#0": {
                    "source": "property",
                    "type": "string",
                    "used": true,
                    "value": "_system.build.datetime" (8)
                }
            }
        },
        "/[CircuitContainer]name=Hello World/[FilterCircuit]name=Hello World Message/[SetAttributeFilter]name=Network zone": {
            "description": "Network zone the API Gateway is located.",
            "fields": {
                "attributeValue#0": {
                    "source": "value"
                    "type": "string",
                    "used": true,
                    "value": "internal" (9)
                }
            }
        },
        "/[CircuitContainer]name=Hello World/[FilterCircuit]name=Hello World Message/[SetAttributeFilter]name=User": {
            "description": "Name of the user building the .fed",
            "fields": {
                "attributeValue#0": {
                    "source": "env" (10)
                    "type": "string",
                    "used": true,
                    "value": "USERNAME" (11)
                }
            }
        },
        "/[CircuitContainer]name=Hello World/[FilterCircuit]name=Hello World Message/[SetAttributeFilter]name=Secret": {
            "description": "Some secret information",
            "fields": {
                "attributeValue#0": {
                    "source": "secrets" (12)
                    "type": "string",
                    "used": true,
                    "value": "secret" (13)
                }
            }
        }
    }
    "properties": { (14)
        "foobar": "myvalue"
    }
}

  1. Environmentalized entities are configured under an entities attribute.

  2. Short hand key of the environmentalized entity.

  3. Name and index of the environmentalized field.

  4. Declares a property (named foobar, see "value") as the source of the field value.

  5. Type of the field (just for documentation, don’t change it).

  6. Indicates if the configured field is used. If false the field is no longer environmentalized or the entity is renamed or removed. The property is automatically maintained by the plugin.

  7. Name of the property containing the value.

  8. Use value of the predefined system property _system.build.datetime.

  9. Literal value for the field.

  10. Specifies an environment variable as the source for the field value.

  11. Field value is retrieved from the USERNAME environment variable.

  12. Specifies an property from the secrets file as the source for the field value.

  13. Field value is retrieved from the secret property in the secrets file.

  14. An optional local definition of properties. If the same property is defined in a separate property file (see below), the separate property has precedence.

📎
 If value is null the field is treated as undefined and the build will fail.

Certificates

The configuration for certificates is stored in a separate JSON file. It specifies the alias of the certificates within the project and the source of the replacement certificate.

When the file is specified it will be automatically generated/updated according to the certificates stored in your project.

src/main/axwgw/gateway.certs.json
{
    "certificates": { (1)
        "extern-crt": { (2)
            "origin": { (3)
                "info": {
                    "not_after": "2020-05-21T07:04:00+02:00", (4)
                    "subject": "CN=extern, O=Axway, L=Berlin, ST=Berlin, C=DE" (5)
                }
            },
            "update": { (6)
                "file": "cert/extern.crt", (7)
                "type": "crt" (8)
            }
        },
        "server-p12": {
            "origin": {
                "info": {
                    "not_after": "2020-05-21T07:02:00+02:00",
                    "subject": "CN=server, O=Axway, L=Berlin, ST=Berlin, C=DE"
                }
            },
            "update": {
                "file": "cert/server.p12",
                "password": "server", (9)
                "source": "password", (10)
                "type": "p12"
            }
        },
        "test": {
            "origin": {
                "info": {
                    "not_after": "2021-09-30T16:01:15+02:00",
                    "subject": "CN=DST Root CA X3, O=Digital Signature Trust Co."
                }
            },
            "update": null (11)
        },
        "test2": { (12)
            "update": {
                "file": "cert/server.p12",
                "password": "server.password", (13)
                "source": "property", (14)
                "type": "p12"
            }
        },
        "test3": {
            "update": {
                "file": "cert/server.p12",
                "password": "TEST3_PASSWORD", (15)
                "source": "env", (16)
                "type": "p12"
            }
        },
        "test4": {
            "origin": {
                "info": {
                    "not_after": "2021-09-30T16:01:15+02:00",
                    "subject": "CN=DST Root CA X3, O=Digital Signature Trust Co."
                }
            },
            "update": {
                "type": "empty" (17)
            }
        }
    }
}

  1. Certificates are configured under a certificates attribute.

  2. Unique alias for storing the certificate in the certificate store.

  3. Information of the origin certificate. A missing origin attribute indicates the origin certificate store doesn’t has a certificate with this alias.

  4. Expiration date of the origin certificate.

  5. Subject of the origin certificate.

  6. Defines the certificate to update the certificate with the same alias within the certificate store. A missing update attribute indicates a new/unconfigured certificate.

  7. Path to the new certificate file.

  8. Type of the certificate. crt for certificates and p12 for certificates with key.

  9. Literal password for the .p12 file.

  10. Declares a property (server.password, see "password") as the source for the password.

  11. null value indicates that the certificate will not be updated.

  12. Certificate without a origin attribute. This certificate will be added to the certificate store.

  13. Password for the .p12 file is retrieved from the property configuration file.

  14. Declares the property "password" as the source of the password for the .p12 file.

  15. The password is retrieved from the TEST3_PASSWORD environment variable.

  16. Specifies an environment variable as the source of the password.

  17. Type empty indicates that a certificate will be updated with an empty certificate and therefore will be removed.

Properties

Properties used in the configuration files for environmentalized fields and certificates can be resolved by a separate configuration file. The file is a JSON document containing a properties property. For each property a name/value pair exists.

gateway.props.json
{
    "properties": { (1)
        "name1": "value1", (2)
        "name2": "value2"
    }
}

  1. Properties are configured under a properties attribute.

  2. For each configured property a name/value pair has to exist. The property is identified by its name.

📎

Properties may contain sensitive information (e.g. credentials). The shift of properties to a separate file enables them to be exclude from the source code management system.

In productive environments secretes may be stored in a secured configuration database. For the build process the property file may be temporarily generated from the configuration database.

Secrets

A secrets file is used to store confidential configurations (e.g. passwords). The values of the properties are encrypted and can be access with a key only. All values are encrypted with the same key.

gateway.crypt.json
{
  "secrets": { (1)
    "__": "3QjoMSfhSelmvMlvcgCdyHf+oTyVnHlyneA3stpN0iQKJ1BUIrY9OA==", (2)
    "my.password": "encrypt:changeme", (3)
    "cert.password": "eL5+ogfSxQue8+NA0/l859g/2nTFwxBUp/7l7z/sMOE=" (4)
  }
}

  1. The secrets property is required.

  2. Marker to check the key. Don’t delete or change it.

  3. The prefix encrypt: indicates that the value changeme has to be encrypted by the encrypt tool.

  4. Values without the prefix are already encrypted.

1. Files will be updated by plugin or config tool automatically