The answer is a chef recipe, but now you’ve got some more questions…

Have a bit of configuration that needs to be addressed on an instance each time it comes up? If you’ve checked with our support team you’ll likely have heard that chef is the route forward, but what is this chef stuff all about anyways? Chef is an automation framework that relies on ‘cookbooks’ and ‘recipes’ which contain a set of tasks to be performed. Chef can do things such as report a message to the Engine Yard dashboard, create a customized configuration file, restart services, call a custom task, or just about anything else you can do from the command line.

For more on the history of chef see this tutorial we’ve started.

Chef on Engine Yard Cloud - Part 1 from Engine Yard on Vimeo.

Chef recipes are written in way that is very similar to Ruby. If you’re already familiar with the language you’ve got a leg up, your ruby knowledge can be used to work minor miracles in chef. If you’re not already familiar with ruby that’s ok, with our example recipes and our video tutorials you’ll have enough information to be dangerous.

So about your staging environment…This post is intended as an introduction to some new concepts. Even with experience, sometimes an idea for a new recipe doesn’t act exactly as expected. A good way to avoid major setbacks is not running your recipes for the first time in your production environment. Instead set up a staging environment to try things out on.

If you’ve already got a replica of your production site set up for staging purposes you have the sincere appreciation of the Engine Yard Support team!! If not, they are not difficult to set up, and can be spun down once you are done with your testing, our hearts go out to you!!

If you need help with setting up a staging environment, it may be easiest to clone the production environment. This article outlines the processes of cloning an existing environment and if you need further assistance, our support team is always available. Alternately, you could boot a new environment for use as a staging platform, though this is not always the best way to ensure things will work in production.

Preparing the Kitchen

There are a few things you’ll want to have on your local computer to effectively work with chef:

1) A text editor

• We recommend vim or Textmate, but any text editor that supports highlighting in .rb files and you are comfortable with is fine.

2) The Engine Yard gem

• This gem provides quite a few features for interacting with the engineyard deploy processes via the command line interface. We’ll be using the gem later to upload the chef recipes once they have been crafted. The engineyard gem can be installed with

  gem install engineyard

3) A fork of Engine Yard’s ey-cloud-recipes cookbook.

• This cookbook contains some helper methods that can be used within your own recipes as well as recipes for many common tasks that can be tweaked and enabled if they suit your needs. For more information on forking a repo, see Github’s help documentation.

4) Get familiar with Ruby and Chef Syntax

• There are many resources out there for familiarizing yourself with the way chef is typed. We recommend LearnChef if you are just getting started or are coming from a non-Ruby language. This will help you to get comfortable with language and syntax and then you’ll be ready to recipe up!

Know Your Ingredients

Chef is aware of quite a few useful bits of information about each instance that it runs on as well as the environment as a whole. This makes it a perfect choice when you’d like one thing configured on utility instance A but not utility instance B, or when you’d like to make one instance aware of a particular setting of another instance such as it’s hostname.

The Node object in chef is what will allow you to access this environment specific data. If you look over our ey-cloud-recipes you’ll see many examples of using the node object peppered throughout.

But what information can you pull from the node object? Quite a bit actually. To see everything that is available, you’ll need to check out a couple of files on a instance. These steps will guide you there.

1) SSH to your instance

2) run sudo cp /etc/chef/dna.json ~/dna.json & sudo chown deploy:deploy ~/dna.json

3) run sudo /usr/local/ey_resin/bin/ohai > ~/ohai.json & sudo chown deploy:deploy ~/ohai.json

• the above commands assume you’re deploy user is named ‘deploy’ which is our default, if you’ve changed this adjust the above commands to reflect this.

These files contain all of the information available to the node object during a chef run. They are JSON formated so you can parse them in your preferred method, or just look at them in a text viewer. The node object is a great way to get your recipe seasoned to your tastes.

Time to Get Cooking

With all this prep out of the way, it’s time to finally get down to the business of making a chef recipe happen. The following steps outline what needs to be done:

note: for this example, we are creating a recipe that outputs some potentially useful data into a test file on the app master instance. The same basic principles can be applied to a wide range of situations

1) In your local environment, in the ey-cloud-recipes repository you’ve created from the fork of the original repo, start with a rake task:

$ rake new_cookbook COOKBOOK=node_knows

This will generate the default file structure needed for a chef cookbook. More information on the file structure of chef can be found here

2) Edit cookbooks/node_knows/recipes/default.rb adding the following

node["applications"].each do |app_name|
  if ['app_master','app','db_master','db_slave','util'].include? node[:instance_role]
    template "/data/#{app_name}/node_knows.txt" do
      source "node_knows.erb"
      owner node[:owner_name]
      group node[:owner_name]
      mode "0644"
      variables({
         :DeployUser => node[:owner_name],
         :EnvName => node[:environment][:name],
         :AppName => app_name,
         :DbName => node[:engineyard][:environment][:apps].find{|d| d[:name] == app_name}[:database_name],
         :AppMasterHostname => node[:engineyard][:environment][:instances].select{|am| am[:role] == 'app_master'}.first[:public_hostname],
         :DbHostname => node['db_host'],
      })
    end
  elsif ['solo'].include? node[:instance_role]
    template "/data/#{app_name}/node_knows.txt" do
      source "node_knows.erb"
      owner node[:owner_name]
      group node[:owner_name]
      mode "0644"
      variables({
        :DeployUser => node[:owner_name],
        :EnvName => node[:environment][:name],
        :AppName => app_name,
        :DbName =>  node[:engineyard][:environment][:apps].find{|d| d[:name] == app_name }[:database_name],
        :AppMasterHostname => node[:engineyard][:environment][:instances].select{|am| am[:role] == 'solo'}.first[:public_hostname],
        :DbHostname => node['db_host'],
      })
    end
  end
end

This is the code that will run on each instance when chef is run. The first thing we do is check the instance role. Because we’ve included all possible roles this will run on every instance, however by limiting the roles allowed, you can fine tune which instances run which bits of code. In our example the the code used for a solo instance varies slightly from that used on the other instance types.

We use a template to create a file /data/$app_name/node_knows.txt regardless of the instance type. Values retrieved from the node object are used to populate various fields with information unique to each environment.

3) Create a text file at cookbooks/node_knows/templates/default/node_knows.erb with the following contents

Linux user used for deploying : <%= @DeployUser%>
App Name : <%= @AppName %>
Environment Name : <%= @EnvName %>
App Master Public Hostname : <%= @AppMasterHostname %>
DB Master Public Hostname <%= @DbHostname %>
Database Name : <%= @DbName %>

A template file is a file that contains some static text and has other portions filled in by variables determined during the chef run. Template files are great for customizing configurations with values that may change from one deploy to another, such as your database hostname.

4) Edit cookbook/main/recipes/default.rb to enable the recipe:

include_recipe "node_knows"

When chef is run it will check the cookbook/main/recipes/default.rb file to find out which other recipes to run. If it’s not included in this file, it won’t be run.

5) Test it out with rake:

$ rake test

This will ensure that no syntax errors have cropped up. If you’ve got a syntax error in a recipe it can cause chef to fail even if the recipe is not required in cookbook/main/recipes/default.rb

6) Once you know everything is working as expected, commit your changes locally and push them to your chef repository (chef recipes do not need to be part of your application repository).

7) Use the ey recipes commands to upload and apply the recipes in the cloned/staging environment you created earlier:

ey recipes upload -e environment_name

ey recipes apply -e environment_name

The ey recipes upload will upload the recipes to a s3 bucket that we maintain. When an apply is run we will pull the recipes from this bucket. This means that changes made to the recipes on the instances, or changes made locally that have not been uploaded with ey recipes upload will not be used. The ey recipes apply is equivilant to pressing the Apply button in the dashboard.

To check if everything has run as expected look for a file named node_knows.txt in /data/App_Name. It should contain information about your environment. Once everything is in place and properly tested and working in staging, you can make the same changes to the production repository and voila! You are a chef recipe mixing machine!!

Ask the Executive Chefs

If the idea of recipe creation seems too daunting a task, there is always the option of contacting our Professional Services team. Provide them with your specs and they can turn out the recipe you need.