Encrypted Data Bags on Cloud on AWS

Many customers have asked us how they should handle shared secrets, passwords and other sensitive data in Chef and now we have a good solution. With the release of our stable-v4 stack, we introduced Chef 10 to the platform, and with it came data bags. Now with the 3.0 release of the engineyard gem, we can officially support data bags and encrypted data bags. You may be wondering what data bags are, how data bags work or how to implement data bags. This blog post will walk you through the entire process.

About Data Bags

Data bags are essentially global JSON data that can be accessed within a recipe. Data bags are indexed and can be searched easily. Encrypted data bags are just data bags encrypted with a secret key.

In a Chef server setup, data bags are created locally and uploaded to the server via Knife. On our Cloud platforms, we use chef-solo, so data bags work a bit differently. Data bags must be uploaded to each chef-solo instance and stored in the directory specified in the /etc/chef-custom/solo.rb file. Functionality to tar and upload your data bags along with your cookbooks has been added to the engineyard gem as of the 3.0 release, so you don’t need to worry about putting the data bags on your instances. You will, however, need to upload the encryption key to all of the instances. In this example, we will use SCP to transfer the key.

Implementing Data Bags

Getting Started

Before you get started, make sure you have version 3.0 or greater of the engineyard gem installed locally and an environment booted on Cloud on AWS.

First, install the same version of Chef that we use on our Cloud on AWS platform onto your local machine:

gem install chef -v 10.16.4

Next, install the knife-solo_data_bag gem. This gem allows us to use Knife to create and manage data bags for use by chef-solo:

gem install knife-solo_data_bag

You will also need to make sure you have a ~/.chef/knife.rb file in place containing the following configurations, replacing the with the full path to your cookbooks:

node_name           “solo”
data_bag_path       "<cookbook_path>/data_bags"

Lastly, create a key file that will be used to encrypt the data bags:

openssl rand -base64 512 > ~/.chef/encrypted_data_bag_secret

Creating a Data Bag

You will want to perform all of the following steps in the root of your cookbooks.

First, create the JSON file containing any shared secrets, replacing with your application name:

echo "{\"id\": \"<application>\", \"api_key\": \"this_is_my_api_key\"}" >> secrets.json

Then, run the following Knife command to create an encrypted data bag from that JSON file, again replacing with your application name:

EDITOR=vi knife solo data bag create secrets <application> --json-file <path to json file> --secret-file ~/.chef/encrypted_data_bag_secret

Lastly, view your data bag to ensure it is encrypted:

knife solo data bag show secrets

Creating a Recipe to Access Your Data Bag

Add the following code to any recipe in your cookbooks:

node[:applications].each do |app_name, _|`
  test_creds = Chef::EncryptedDataBagItem.load "secrets", app_name
  Chef::Log.info "API Key: #{test_creds["api_key"]}"
end

Uploading the Data Bag and Encryption Key

You can upload your encryption key using the following bash command, replacing with your environment name and with your account name:

for server in `ey servers -Su -e <environment> --account=<account>` ; do scp -o StrictHostKeyChecking=no ~/.chef/encrypted_data_bag_secret $server:/home/deploy/; ssh -o StrictHostKeyChecking=no $server 'sudo mv /home/deploy/encrypted_data_bag_secret /etc/chef/';  done

Next, upload and apply the cookbooks, again replacing with your environment name and with your account name:

ey recipes upload --apply -e <environment> --account <account>

Finally, verify that the data bag was successfully decrypted by viewing the custom Chef logs found in the dashboard and ensuring that the API key logged is the same as the API key that you encrypted.

Conclusion

This solution is a big improvement over storing unencrypted secrets in your cookbook attributes, but it’s not entirely secure. We still have to transfer the encryption key up to the instances and while this is done via SCP, it’s not optimal to have to send that file across the Internet. In another blog post, we will go over an even better solution: using chef-vault and your own Chef server.