chef dynamically decrypt databags
We use Chef databags to store sensitive data necessary in the build / deployment of our systems.
When a databag is uploaded to Chef, it is uploaded in encrypted format, and the plain text version is only available on the machine that generated the databag.
We use git to store and version control our encrypted databags, but since we of course do not want our unencrypted secrets in git, we need a way to be able to access the secrets generated by another engineer and running on our systems.
To solve this, two scripts were created which enable us to iterate through all of the encrypted databags, and decrypt them into a
data_bags_unencrypted
directory.
From here, we can then modify the plain-text databags and then re-encrypt and re-upload them to the Chef server as usual.
decrypt-databag.rb
require 'chef/encrypted_data_bag_item'
require 'json'
keyfile = ARGV[0]
encrypted_path = ARGV[1]
if encrypted_path == nil
puts "Encrypted Databag Path REQUIRED"
exit 1
end
if not File.file?(keyfile)
puts "keyfile REQUIRED"
exit 1
end
if not File.file?(encrypted_path)
puts "encrypted_path REQUIRED"
exit 1
end
secret = Chef::EncryptedDataBagItem.load_secret(keyfile)
encrypted_data = JSON.parse(File.read(encrypted_path))
plain_data = Chef::EncryptedDataBagItem.new(encrypted_data, secret).to_hash
puts JSON.pretty_generate(plain_data)
decrypt-databags.sh
#!/bin/bash
set -e
if [[ $OSTYPE == msys* ]]; then
CHEF_RUBY=C:/opscode/chefdk/embedded/bin/ruby
else
CHEF_RUBY=/opt/chefdk/embedded/bin/ruby
fi
CHEF_DIR=/path/to/chef/data
DECRYPT_SCRIPT_PATH=/path/to/decrypt-databag.rb
UNENCRYPTED_DATA=$CHEF_DIR/data_bags_unencrypted
KEYFILE=$UNENCRYPTED_DATA/encrypted_data_bag_secret
ENCRYPTED_DATABAGS=$CHEF_DIR/data_bags
cd $ENCRYPTED_DATABAGS
git pull
check_dependencies () {
if [ ! -f $CHEF_RUBY ]; then
echo $CHEF_RUBY REQUIRED
exit 1
fi
if [ ! -f $DECRYPT_SCRIPT_PATH ]; then
echo $DECRYPT_SCRIPT_PATH REQUIRED
exit 1
fi
if [ ! -d $UNENCRYPTED_DATA ]; then
echo $UNENCRYPTED_DATA REQUIRED
exit 1
fi
if [ ! -f $KEYFILE ]; then
echo $KEYFILE REQUIRED
exit 1
fi
if [ ! -d $ENCRYPTED_DATABAGS ]; then
echo $ENCRYPTED_DATABAGS REQUIRED
exit 1
fi
}
loop_directories () {
for f in $(find . -maxdepth 1 -type d); do
loop_files "$f"
done
}
loop_files () {
for f in $(ls $1/*.json 2>/dev/null); do
chef_decrypt_databag "$f"
done
}
save_existing_databags () {
pushd $CHEF_DIR
BAKDIR=$CHEF_DIR/data_bags_backup
mkdir -p $BAKDIR
BAKFILE=$BAKDIR/data_bags_unencrypted-bak_`date +%F_%T | sed -e 's,/,-,g' -e 's,:,-,g'`.zip
if [[ $OSTYPE == msys* ]]; then
"C:\Program Files\7-Zip\7z.exe" a $BAKFILE data_bags_unencrypted/*.*
else
zip -r $BAKFILE data_bags_unencrypted
fi
popd
}
chef_decrypt_databag () {
DIR_NAME=$(dirname "$1" | sed 's,./,,g')
FILE_NAME=$(basename "$1")
UNENCRYPTED_PATH=$UNENCRYPTED_DATA/$DIR_NAME/$FILE_NAME
mkdir -p "$UNENCRYPTED_DATA/$DIR_NAME"
touch "$UNENCRYPTED_PATH"
echo Decrypting $DIR_NAME/$FILE_NAME
$CHEF_RUBY $DECRYPT_SCRIPT_PATH $KEYFILE "$1" > $UNENCRYPTED_PATH
}
main () {
check_dependencies
save_existing_databags
loop_directories
}
main
So what is this all doing? The first script -
decrypt-databag.rb
- is necessary to use Chef to decrypt the encrypted databags.
If you call this script (using Chef's provided Ruby) with the path to your keyfile and path to the encrypted data bag item, it will decrypt it for you.
Great, but we have hundreds of databags, not to mention multiple items within each databag. That's where the second script helps.
Of course this script will have to be modified to point to your Chef data directory, keyfile, etc. All the config is in the first few lines, once you see "git pull", it's all parameterized from that point on.
The script was written to run on both *NIX and Git Bash in Windows. On *NIX, you need
zip
installed. On Windows, you need
7-Zip
.
This is because before we modify your existing databags in situ, we zip them up in a backup directory.
We have this script set up to be called as a
post-merge
git hook in our encrypted data bag repo.
This ensures that every time there is a change to the encrypted data bags, when we pull the changes, the updated databags are automatically decrypted and available for review / modification.
last updated 2024-03-18