-
Notifications
You must be signed in to change notification settings - Fork 5
SOPS
SOPS is a piece of software for storing GPG encrypted secrets in your repository. Your controller server setup steps will have created a GPG (more information on Wikipedia) key for your controller user, published it to a set of key servers and installed SOPS for you. The SOPS README on GitHub is very detailed, this page is intended as a quick start for SOPS and using SOPS with Ansible.
Unfortunately Windows is not supported by the SOPS project. If you need to use a Windows workstation, we recommend you run Linux on a virtual machine or a container.
You will need to make sure you have GPG tools installed on your workstation. Binaries are available from the project, most Linux distributions will install GPG by default, other operating systems have suggested projects on that same download page. Mac users can also use Homebrew. The SOPS project ships ready installers for most Linux distributions and Mac on their GitHub releases page. Just download the appropriate installer/package and run/install it.
The GPG project has a good guide on generating a new keypair, if you do not have one. Here is an excellent 'gist' on GitHub for migrating keys from one machine to another. Once you have created a key, you need to publish it to a key server, so it can be easily shared with your colleagues:
# List the local GPG secret keys, to obtain the ID of the key
gpg --list-secret-keys
# Replace the ID with the ID of the key to upload
gpg --keyserver hkps://keyserver.ubuntu.com --send-keys 415BEDA28728018B7EA3B15E9AEE4AEA16CE8B83Note, key servers are obsolete however without them the alternative is sending public keys to each other over email, or some other equally awful mechanism, so as long as someone is still running working ones we prefer to use them. We predominantly use the Ubuntu key server these days, because it is still running and stable.
As mentioned in the intro, ce-provision created a GPG key for your server. To retrieve it, login to your server terminal (SSH or browser) and run gpg --list-secret-keys to get the fingerprint ID. Copy this and then, on your workstation again, run these commands:
# Verify the key server has the key you want - press Q to exit
gpg --keyserver keyserver.ubuntu.com --search-keys 415BEDA28728018B7EA3B15E9AEE4AEA16CE8B83
# Fetch the key
gpg --keyserver keyserver.ubuntu.com --recv-keys 415BEDA28728018B7EA3B15E9AEE4AEA16CE8B83The key will be added into GPG and will be ready to use at this point.
You can use SOPS anywhere, but we principally deploy it to our ce-provision-config repository. If you followed our Quick Start guides then you will already have an example .sops.yaml in your config repository, which will look like this:
---
creation_rules:
- key_groups:
- pgp:
- XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX # your PGP key fingerprints here as a YAML listYou can get a full list of available public keys on your machine at this point with this command:
gpg --list-keysAdd them to the YAML file like this:
---
creation_rules:
- key_groups:
- pgp:
- 415BEDA28728018B7EA3B15E9AEE4AEA16CE8B83 # my server
- E44E4AE251C887AD87A2FADEBBA1236AD9F94E72 # my workstationYou can also add the GPG key fingerprints of colleagues, by asking them to follow the above create step, share their key ID with you and then fetching their keys, in the same way as you fetched the server's key. Once that file is saved you are ready to create encrypted variables. You can use SOPS just like any other CLI editor, you just type sops followed by a filename.
SOPS encrypted files can be named anything, however the SOPS plugin for Ansible, which is normally enabled by default in ansible.cfg if you followed our Quick Start, expects a specific naming convention. All files must be named _encrypted.sops.yml.
Still in your ce-provision-config repo, run this command:
sops hosts/group_vars/all/_encrypted.sops.ymlSOPS will open your default system CLI editor and, if this is a new file, it will pre-fill it with some examples. You can delete those and add any valid YAML you like, for example:
_gitlab_root_password: asdf1234qwer5678Save and exit in the usual way and the file will be created, but you will notice if you open it in your editor directly the value will have been encrypted and key information appended. Add the file to your repository and push it up and it will be ready to use. Because we put it in the all directory of our group vars in ce-provision-config it will be available to all servers. Similarly, had we placed it at hosts/group_vars/_controller then it would only be available to servers in the _controller host group.
You can now use the variable _gitlab_root_password in Ansible as you could with any other variable, because at runtime Ansible will decrypt any _encrypted.sops.yml files it finds in its scope and load the variables, for example I can now do this in a playbook and Ansible will be able to load and use the value:
vars:
gitlab:
initial_root_password: "{{ _gitlab_root_password }}"If you add or remove GPG keys after the initial creation of your encrypted files, you need to do the following:
- Add (or remove) the key fingerprint from the .sops.yaml file at the root of the repo
- Re-encrypt all files:
find . -name '_encrypted.sops.yml' -exec sops updatekeys {} \; If you no longer need/want a key, you can delete it locally like this:
gpg --delete-key 415BEDA28728018B7EA3B15E9AEE4AEA16CE8B83
Be careful not to do so until you are sure it is no longer present in any of your .sops.yaml files and that all your encrypted files have been updated.
If when running sops _encrypted.sops.yml file you might get the error Failed to get the data key required to decrypt the SOPS file. Try setting this environment variable as you may have a gpg agent running which requires it:
GPG_TTY=$(tty)
export GPG_TTY
If you see this output:
gpg: agent_genkey failed: Permission denied
Key generation failed: Permission denied
You most likely need to do this: https://github.com/MISP/MISP/issues/2372#issuecomment-452786029