Secrets with Ansible: Ansible Vault and GPG

I was blown away last night at our Ansible PDX meetup by a great presentation by Andrew Lorente about how to track secrets with your applications.  Andrew gave a method of how to do this that I wanted to write down so I know how to do it.  Andrew has his own blog here where he wrote about the solution.  I wanted to go over it a little more in details cause I want to make sure it sticks in my head!  (bonus: I also learned about the pbcopy command on Mac last night!)  The other thing is since I didn't have any of this on my machine it helps someone get started who hasn't done anything with GPG yet. His technique involves some pretty simple tools:

• Ansible Vault
• GPG / GPG-Agent

1. Generate an Ansible Vault Password

On my mac I run:

brew install pwgen
brew install gpg
brew install gpg-agent

This gets me my tools!  Ok, so now I need to generate my pgp key.

gpg --gen-key

Doing this I just accepted all the defaults. Now I generate a password for the vault.

pwgen -cynC | head -1 | gpg -e -o vault_passphrase.gpg
You did not specify a user ID. (you may use "-r")

Current recipients:

Enter the user ID.  End with an empty line: vallard
                                                    
Current recipients:
2048R/BDF6142D 2015-02-27 "Vallard Benincosa <vallard@benincosa.com>"

Enter the user ID.  End with an empty line:

Now that I have that I follow Andrew's instructions and create a file called open_the_vault.sh with the contents being:

#!/bin/sh
gpg --batch --use-agent --decrypt vault_passphrase.gpg

Then make sure I can run this file as an executable

chmod +x open_the_vault.sh

Add this to my ansible.cfg file

[defaults]
vault_password_file=open_the_vault.sh
hostfile = ./inventory
host_key_checking = false

 2.  Setup the GPG Agent

If you now run the command ./open_the_vault.sh you'll find that it says: "Hey, there's no agent running!".  There are a few ways we can start the agent.  You can create a LaunchAgent as shown here, or you can just configure something in your own shell.  I went with my own shell method and basically followed this post. Create ~/.bash_gpg

envfile="${HOME}/.gnupg/gpg-agent.env"

if test -f "$envfile" && kill -0 $(grep GPG_AGENT_INFO "$envfile" | cut -d: -f 2) 2>/dev/null; then
    eval "$(cat "$envfile")"
else
    eval "$(gpg-agent --daemon --log-file=~/.gpg/gpg.log --write-env-file "$envfile")"
fi
export GPG_AGENT_INFO  # the env file does not contain the export statement

Append to ~/.bashrc

GPG_AGENT=$(which gpg-agent)
GPG_TTY=`tty`
export GPG_TTY

if [ -f ${GPG_AGENT} ]; then
    . ~/.bash_gpg
fi

Create ~/.gnupg/gpg-agent.conf

default-cache-ttl 600
pinentry-program /usr/local/bin/pinentry
no-grab
max-cache-ttl 172800

Opening up a new shell, I should now be able to run  the ./open_the_vault.sh command.  It will ask me for my password the first time, but if I run it again, it won't ask me again.  Right now the default-cache-ttl is set to 600 or 10 minutes.  This can be increased if I want it open longer.

3. Encrypt the file

The file I will be encrypting is a main.yml file that contains all my variables.  Since it already exists I run the command

ansible-vault encrypt roles/openstack-controller/vars/main.yml

Now, if I look at this file roles/openstack-controller/vars/main.yml you'll see its just a bunch of random encypted numbers!  Awesome!  Now my environment variables and my password file can all be committed with git. Now obviously, if you look at the history of this project I'm working on, you'll see the old unencrypted file, but that's ok, I've changed the passwords now so its super secure!  From now on though, no more simple passwords and I'll be using these methods to encrypt.

4.  Sharing Keys

Obviously this solution works great for one developer, but what if we have more developers?  They will also need to be able to run the key.  To do this, we just need to encrypt the file with all of our users.  We now decrypt the vault_passphrase.gpg with our open_the_vault.sh command.  We then get the output of our passphrase.

./open_the_vault.sh >unencypted_passphrase

Now, we encrypt it again with all of our users.  The new user will need to share his key with you so that you can encrypt it.

cat unencypted_passphrase | gpg -e -o vault_passphrase.gpg

Test that it works by trying to edit the file

ansible-vault edit roles/openstack-controller/vars/main.yml