Skip to content

Commit

Permalink
Merge pull request #40 from Shopify/docs-debt
Browse files Browse the repository at this point in the history 
docs: sops and datadog
  • Loading branch information
@thepwagner
thepwagner authored Sep 14, 2021
2 parents 7c4e841 + 473280b commit 6956736
Show file tree
Hide file tree
Showing 3 changed files with 99 additions and 19 deletions.
26 changes: 20 additions & 6 deletions v2/cmd/voucher_server/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -56,14 +56,30 @@ Below are the configuration options for Voucher Server:
| `server` | `password` | A password hashed with the bcrypt algorithm, for use with the username. |
| `ejson` | `dir` | The path to the ejson keys directory. |
| `ejson` | `secrets` | The path to the ejson secrets. |
| `sops` | `file` | The path to the SOPS secrets. |
| `clair` | `address` | The hostname that Clair exists at. If "http://" or "https://" is omitted, this will default to HTTPS. |
| `repository.[alias]` | `org-url` | The URL used to determine if a repository is owned by an organization. |
| `required.[env]` | (test name here) | A test that is active when running "env" tests. |
| `statsd` | `backend` | The destination for reporting metrics, can be `statsd` for local aggregation, or `datadog`. |
| `statsd` | `addr` | The UDP endpoint to use when `statsd.backend == "statsd"`. |
| `statsd` | `tags` | List of tags in `key:value` format that apply to every metric. Example: `env:production`. |
| `statsd` | `sample_rate` | Configurable sample rate to limit metrics overhead. |

Configuration options can be overridden at runtime by setting the appropriate flag. For example, if you set the "port" flag when running `voucher_server`, that value will override whatever is in the configuration.

Note that Repositories can be set multiple times. This is discussed futher below.

Secret values are stored encrypted in an ejson or SOPS file, and can not be overwritten at the command line. Below are the configuration secrets for Voucher Server. Ejson and SOPS secrets follow the same schema:

| Group | Key | Description |
| :------------- | :--------------------------- | :---------------------------------------------------------------------------------------------------- |
| `openpgpkeys` | (test name here) | The PGP key to use for signing attestations of a specific test. |
| `clair` | `username` | Username for CoreOS Clair, if configured as a scanner. |
| `clair` | `password` | Password for CoreOS Clair, if configured as a scanner. |
| `datadog` | `api_key` | API key for direct submission when configuration `statsd.backend == "datadog"`. |
| `datadog` | `app_key` | App key for direct submission when configuration `statsd.backend == "datadog"`. |
| `repositories` | (repository owner name here) | Credentials for repository authentication. |

### Scanner

The `scanner` option in the configuration is used to select the Vulnerability scanner.
Expand Down Expand Up @@ -156,14 +172,13 @@ org-url = "https://github.com/grafeas"
#### Repository Authentication

For the repository groups to authenticate with a repository server, you will
need to create secrets matching the alias name in your `ejson` file.
need to create secrets matching the alias name in your secrets file (ejson or SOPS).

For example, for the `repository.shopify` block, you would need a `repositories`
block in your `ejson` with a `shopify` block inside of it:
block in your secrets with a `shopify` block inside of it:

```json
{
"_public_key" : "cb0849626842a90427a026dc78ab39f8ded6f3180477c848ed3fe7c9c85da93d",
"repositories": {
"shopify": {
"token": "EJ[1:vt978NEKynsJhZlCvg5XdSE2S2PM1JBPK0tlC05cQAc=:4mDZCykfieedtHGoM0UT+Wr6zPO9J6XO:/AuGJ3I2QVnk52qOLo0sQ+EzEAk=]"
Expand All @@ -183,7 +198,6 @@ you can specify the App ID, Installation ID, and the private key as follows:

```json
{
"_public_key" : "cb0849626842a90427a026dc78ab39f8ded6f3180477c848ed3fe7c9c85da93d",
"repositories": {
"shopify": {
"_id": "1234",
Expand Down Expand Up @@ -264,15 +278,15 @@ ignored unless called directly.

#### OpenPGP Keys

You can use OpenPGP keys by encrypting them in your ejson secrets file.
You can use OpenPGP keys by encrypting them in your secrets file (ejson or SOPS).

First, ensure the signer is set to `pgp` in the configuration:

```toml
signer = "pgp"
```

Then add keys to your ejson secrets file. [This is documented more completely
Then add keys to your secrets file. [This is documented more completely
in the Tutorial](TUTORIAL.md#generating-keys-for-attestation).

#### Google KMS Keys
Expand Down
82 changes: 74 additions & 8 deletions v2/cmd/voucher_server/TUTORIAL.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,13 @@ First, install Voucher using the instructions in the [README](/v2/cmd/voucher_se

Create a new configuration file. You may want to refer to the [example configuration](/config/config.toml).

Make sure that you update the `ejson` specific blocks to point to the `ejson` file and key you will be making in the next step.
Make sure that you update the `ejson` or `sops` specific blocks to point to the files you will be making in the next step.

## Create ejson configuration
## Create secrets

If you plan on creating attestations (rather than just running checks against your images), or if you plan on using Clair as your Vulnerability Scanner, you will need to create an `ejson` file to store the OpenPGP keys and/or Clair login information respectively.
If you plan on creating attestations (rather than just running checks against your images), or if you plan on using Clair as your Vulnerability Scanner, you will need to create a secrets file to store secret values like the OpenPGP keys and/or Clair login information.

### Create ejson configuration

Note: this step is unnecessary if you are a Shopify employee and are running Voucher in Shopify's cloud platform.

Expand Down Expand Up @@ -57,6 +59,64 @@ You can now decrypt your `ejson` file using:
$ ejson --keydir=<path to the directory containing the keyfile> decrypt <filename of your ejson file>
```

### Create SOPS configuration

First, establish what key you'd like SOPS to use. See [SOPS usage](https://github.com/mozilla/sops#usage) for the options supported.
Voucher works best with external key storage, such as your cloud provider's KMS. This example will use the Google [Cloud Key Management](https://cloud.google.com/security-key-management) service.

You can create a `json` file containing the secrets in plaintext. For example:

```json
{
"openpgpkeys": {
"diy": "-----BEGIN PGP PRIVATE KEY BLOCK-----\n..."
}
}
```

Follow your provider's instructions to [create a new key](https://cloud.google.com/kms/docs/creating-keys). This example uses:
* Project name: `my-awesome-project`
* Keyring name: `voucher`
* Key name: `voucher-sops`

You can then encrypt the file using the KMS key, and verify the result is encrypted:
```shell
$ sops -e --gcp-kms projects/my-awesome-project/locations/global/keyRings/voucher/cryptoKeys/voucher-sops config/secrets-plaintext.production.json > config/secrets.production.json
$ cat config/secrets.production.json
{
"openpgpkeys": {
"diy": "ENC[AES256_GCM,data:zcNXbuPAkZMuer+sXqg4F1wAblKzm93K76c0WIU=,iv:ajzwCjEaT+sPW30LrHT+F7m7tSmJDfL5AEBfU6DU7a0=,tag:ILlBpERerNBRxC8VQ7xKSw==,type:str]"
},
"sops": {
"kms": null,
"gcp_kms": [
{
"resource_id": "projects/my-awesome-project/locations/global/keyRings/voucher/cryptoKeys/voucher-sops",
"created_at": "2021-09-10T20:47:03Z",
"enc": "CiQAxEdEknNVKrA0FQG0v1FKol9sQNKsaiQerEyXG7ueLz2vBRdeLESCTN0P9D082yxFLF4QGPHtToBrUSSQBF/xdprZIxAqSn2slzIYGBTx+sr+GNy2fEakJP8UYaQDhGjBfVVsRvMwWgFYuKpF4yg="
}
],
"azure_kv": null,
"hc_vault": null,
"age": null,
"lastmodified": "2021-09-10T20:47:03Z",
"mac": "ENC[AES256_GCM,data:DzYveNlcRvhrZigynfCtL4HbHS8VuoKlaozQUZD3UHQwnEraifwAQcZanHWYqW6EWj84YMG2GmT5lGYFJzMe9KTyoDXO6IDFMORDxyGaH1RddFzsn7QyLFttwvxQ+5u/J0xpQTEzzZUAJravHtx+xg4i6W0Uv22FS15HaoFMObZQh+9tJHMjzqVduNN48VkVs=,iv:ldhb/UUqYBoZWUe/SNwo=,tag:G/VgUd/c0JwFJKA3ZmfRBg==,type:str]",
"pgp": null,
"unencrypted_suffix": "_unencrypted",
"version": "3.7.1"
}
}
```

Verify you can decrypt the file, then delete the plaintext copy:
```shell
$ export EDITOR="code -w" # optional, if you prefer vscode
$ sops config/secrets.production.json
$ rm config/secrets-plaintext.production.json
```

In the future if you need to edit the file, use the same `sops config/secrets.production.json` command. SOPS files embed a message authentication code, so you can only edit in this way.

## Generating Keys for Attestation

Attestation uses GPG signing keys to sign the image. It's suggested that you use a primary GPG key instead of a subkey.
Expand Down Expand Up @@ -87,7 +147,7 @@ What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
```

When prompted for how long the key should be valid, you may want to specify that the key does not expire, especially if the team maintaining your Binary Authorization configuration is the same team managing your Voucher install.
When prompted for how long the key should be valid, select an [appropriate cryptoperiod](https://www.keylength.com/) for your deployment. For this tutorial we'll specify that the key does not expire.

```
Please specify how long the key should be valid.
Expand All @@ -101,7 +161,7 @@ Key does not expire at all
Is this correct? (y/N) y
```

You will next be asked to provide an ID for the GPG key. You will want to add a comment to claify which Check this key is for.
You will next be asked to provide an ID for the GPG key. You will want to add a comment to clarify which Check this key is for.

```
GnuPG needs to construct a user ID to identify your key.
Expand Down Expand Up @@ -147,20 +207,26 @@ Our example key from before would then look like this.
-----BEGIN PGP PRIVATE KEY BLOCK-----\nlQcYBFt23t8BEADuZqi....
```

Next, create a new value in the `openpgpkeys` block in your `ejson` file. Make sure the key name is the same as it's name in the source code (eg, for the "DIY" test, use "diy"):
Next, create a new value in the `openpgpkeys` block in your secrets file. Make sure the key name is the same as it's name in the source code (eg, for the "DIY" test, use "diy"):

```json
{
"_public_key": "<public key>",
"openpgpkeys": {
"diy": "-----BEGIN PGP PRIVATE KEY BLOCK-----\nlQcYBFt23t8BEADuZqi...."
},
"clair": {}
}
```

and call `ejson encrypt` to encrypt it:
For ejson, edit the file and call `ejson encrypt` to encrypt it:

```shell
$ code secrets.ejson
$ ejson encrypt secrets.ejson
```

For SOPS, call `sops` to edit the file.

```shell
$ sops secrets.json
```
10 changes: 5 additions & 5 deletions v2/cmd/voucher_subscriber/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,11 @@ $ voucher_subscriber [--project <project> --subscription <subscription>]

`voucher_subscriber` supports the following flags:

| Flag | Short Flag | Description |
| :-------- | :--------------- | :------------------------------------------------------------------------- |
| `--project` | `-p` | The GCP project to be used. |
| `--subscription` | `-s` | The subscription that contains messages. |
| `--timeout` | | The number of seconds to spend checking an image, before failing. |
| Flag | Short Flag | Description |
| :-------- | :----------- | :------------------------------------------------------------------------- |
| `--project` | `-p` | The GCP project to be used. |
| `--subscription` | `-s` | The subscription that contains messages. |
| `--timeout` | | The number of seconds to spend checking an image, before failing. |

For example:

Expand Down

0 comments on commit 6956736

Please sign in to comment.