chef dynamically decrypt databags

We use Chef databags to store sensitive data necessary in the build / deployment of our systems.

When a databag is uploaded to Chef, it is uploaded in encrypted format, and the plain text version is only available on the machine that generated the databag.

We use git to store and version control our encrypted databags, but since we of course do not want our unencrypted secrets in git, we need a way to be able to access the secrets generated by another engineer and running on our systems.

To solve this, two scripts were created which enable us to iterate through all of the encrypted databags, and decrypt them into a
data_bags_unencrypted
directory.

From here, we can then modify the plain-text databags and then re-encrypt and re-upload them to the Chef server as usual.

decrypt-databag.rb
require 'chef/encrypted_data_bag_item' require 'json' keyfile = ARGV[0] encrypted_path = ARGV[1] if encrypted_path == nil puts "Encrypted Databag Path REQUIRED" exit 1 end if not File.file?(keyfile) puts "keyfile REQUIRED" exit 1 end if not File.file?(encrypted_path) puts "encrypted_path REQUIRED" exit 1 end secret = Chef::EncryptedDataBagItem.load_secret(keyfile) encrypted_data = JSON.parse(File.read(encrypted_path)) plain_data = Chef::EncryptedDataBagItem.new(encrypted_data, secret).to_hash puts JSON.pretty_generate(plain_data)

decrypt-databags.sh
#!/bin/bash set -e if [[ $OSTYPE == msys* ]]; then CHEF_RUBY=C:/opscode/chefdk/embedded/bin/ruby else CHEF_RUBY=/opt/chefdk/embedded/bin/ruby fi CHEF_DIR=/path/to/chef/data DECRYPT_SCRIPT_PATH=/path/to/decrypt-databag.rb UNENCRYPTED_DATA=$CHEF_DIR/data_bags_unencrypted KEYFILE=$UNENCRYPTED_DATA/encrypted_data_bag_secret ENCRYPTED_DATABAGS=$CHEF_DIR/data_bags cd $ENCRYPTED_DATABAGS git pull check_dependencies () { if [ ! -f $CHEF_RUBY ]; then echo $CHEF_RUBY REQUIRED exit 1 fi if [ ! -f $DECRYPT_SCRIPT_PATH ]; then echo $DECRYPT_SCRIPT_PATH REQUIRED exit 1 fi if [ ! -d $UNENCRYPTED_DATA ]; then echo $UNENCRYPTED_DATA REQUIRED exit 1 fi if [ ! -f $KEYFILE ]; then echo $KEYFILE REQUIRED exit 1 fi if [ ! -d $ENCRYPTED_DATABAGS ]; then echo $ENCRYPTED_DATABAGS REQUIRED exit 1 fi } loop_directories () { for f in $(find . -maxdepth 1 -type d); do loop_files "$f" done } loop_files () { for f in $(ls $1/*.json 2>/dev/null); do chef_decrypt_databag "$f" done } save_existing_databags () { pushd $CHEF_DIR BAKDIR=$CHEF_DIR/data_bags_backup mkdir -p $BAKDIR BAKFILE=$BAKDIR/data_bags_unencrypted-bak_`date +%F_%T | sed -e 's,/,-,g' -e 's,:,-,g'`.zip if [[ $OSTYPE == msys* ]]; then "C:\Program Files\7-Zip\7z.exe" a $BAKFILE data_bags_unencrypted/*.* else zip -r $BAKFILE data_bags_unencrypted fi popd } chef_decrypt_databag () { DIR_NAME=$(dirname "$1" | sed 's,./,,g') FILE_NAME=$(basename "$1") UNENCRYPTED_PATH=$UNENCRYPTED_DATA/$DIR_NAME/$FILE_NAME mkdir -p "$UNENCRYPTED_DATA/$DIR_NAME" touch "$UNENCRYPTED_PATH" echo Decrypting $DIR_NAME/$FILE_NAME $CHEF_RUBY $DECRYPT_SCRIPT_PATH $KEYFILE "$1" > $UNENCRYPTED_PATH } main () { check_dependencies save_existing_databags loop_directories } main

So what is this all doing? The first script -
decrypt-databag.rb
- is necessary to use Chef to decrypt the encrypted databags.

If you call this script (using Chef's provided Ruby) with the path to your keyfile and path to the encrypted data bag item, it will decrypt it for you.

Great, but we have hundreds of databags, not to mention multiple items within each databag. That's where the second script helps.

Of course this script will have to be modified to point to your Chef data directory, keyfile, etc. All the config is in the first few lines, once you see "git pull", it's all parameterized from that point on.

The script was written to run on both *NIX and Git Bash in Windows. On *NIX, you need
zip
installed. On Windows, you need
7-Zip
.

This is because before we modify your existing databags in situ, we zip them up in a backup directory.

We have this script set up to be called as a
post-merge
git hook in our encrypted data bag repo.

This ensures that every time there is a change to the encrypted data bags, when we pull the changes, the updated databags are automatically decrypted and available for review / modification.

last updated 2019-02-18T15:56:19+0000