Tuesday 3 January 2017

Chapter 7. Cookbook Authoring and Use

Cookbooks are the fundamental component of infrastructure management with Chef. Think of a cookbook as a package for your recipes. Each cookbook represents the set of instructions required to configure or deploy a single unit of infrastructure such as a web server, database, or application. The recipes with code are only a small part of the entire equation. A cookbook also contains any supporting components, such as archives, images, or libraries. In addition, a cookbook holds configuration information, platform-specific implementations, and resource declarations required to manage a piece of infrastructure with Chef.

Your First Cookbook: Message of the Day

For your first cookbook, let’s automate the configuration of a message of the day on our guest node running CentOS 6. Let’s make it unambiguous that you have logged in to the guest node, by using Chef to configure a message on login stating that this is the guest node.
The command to generate an initial cookbook directory structure will differ, depending on whether you installed the Chef Development Kit or Chef Client on your host.
If you have the Chef Development Kit installed, continue on to the next section, Your First Cookbook: Message of the Day (Chef Development Kit), to learn how to create a cookbook using the chef utility. If you have the Chef Client installed, skip ahead to Your First Cookbook: Message of the Day (Chef Client) to learn how to create a cookbook using the knife utility.

Your First Cookbook: Message of the Day (Chef Development Kit)

You’ll be using a tool called chef to generate an initial directory structure for a message of the day cookbook (motd). chef is a new common utility command that debuted with the Chef Development Kit. On a command line, run the chef generate cookbook motd command to create the cookbook directory scaffolding. chef generate will create a main directory for your cookbook called motd as part of the process:
$ chef generate cookbook motd
Compiling Cookbooks...
Recipe: code_generator::cookbook
  * directory[/Users/misheska/learningchef/motd] action create
    - create new directory /Users/misheska/learningchef/motd
...
  * template[/Users/misheska/learningchef/motd/recipes/default.rb] action create
    - create new file /Users/misheska/learningchef/motd/recipes/default.rb
    - update content in file /Users/misheska/learningchef/motd/recipes/default
    .rb from none to 9cc885
    (diff output suppressed by config)
Make the motd directory you just created the current directory:
$ cd motd
As demonstrated in Example 7-1, modify the .kitchen.yml file to use the CentOS 6 image we’ve tailored for the book.
Example 7-1. chefdk/motd/.kitchen.yml
---
driver:
  name: vagrant

provisioner:
  name: chef_zero

platforms:
  - name: centos65
    driver:
      box: learningchef/centos65
      box_url: learningchef/centos65

suites:
  - name: default
    run_list:
      - recipe[motd::default]
    attributes:

NOTE

As of this writing, there is a bug in the Chef Development Kit 0.1.0 where the chef command generates an incorrect reference to recipe[bar::default]. It should be recipe[motd::default] instead, matching the name of the cookbook. This bug is fixed in the Chef Development Kit 0.2.0-2 release, and should be available by the time you read this. If not, fix the recipe line in the .kitchen.yml accordingly.
Check to make sure there are no syntax errors in your kitchen.yml file by running kitchen list. If you see a stack trace error instead of the following output, you likely made a typo, inadvertently used tabs instead of spaces, or didn’t line up entries correctly:
$ kitchen list
Instance          Driver   Provisioner  Last Action
default-centos65  Vagrant  ChefZero     <Not Created>
On CentOS, in order to update the message of the day, we need to create a static text file called /etc/motd on the node. The /etc/motd file will contain the text with our message of the day. We’ll create a copy of the motd file we want created in our cookbook. This is how Chef manages files. Then we’ll add code to our recipe to ensure that the file will be copied from the cookbook to the node in the appropriate location in the /etc directory.
Use the chef generate file motd command to generate the directory structure required for the motd file we will be creating on the node. We need only use the name of the file we want to create, not the path:
$ chef generate file motd
Compiling Cookbooks...
Recipe: code_generator::cookbook_file
  * directory[/Users/misheska/learningchef/motd/files/default] action create
    - create new directory /Users/misheska/learningchef/motd/files/default
  * template[/Users/misheska/learningchef/motd/files/default/motd] action create
    - create new file /Users/misheska/learningchef/motd/files/default/motd
    - update content in file /Users/misheska/learningchef/motd/files/default
    /motd from none to e3b0c4
    (diff output suppressed by config)
With your handy programmer’s text editor, edit the file files/default/motd you just created in your motdcookbook. We think all messages are more effective when spoken by a friendly warning cow, so we added a bit of ASCII art to our file, as shown in Example 7-2.
Example 7-2. chefdk/motd/files/default/motd
  _________________________________________________
  < YOU ARE ON A SIMULATED CHEF NODE ENVIRONMENT! >
   -----------------------------------------------
          \   ^__^
           \  (oo)\_______
              (__))\/\    \
                  ||----w |
                  ||     ||

Introducing the Cookbook_file Resource

We’ll use Chef to help us more easily determine that we are running on a guest virtual machine node by writing some automation to change the Linux message of the day. On Linux, the message of the day is displayed when a user logs in. The message of the day is used by Linux administrators to communicate with users. You can change the message of the day by editing the file /etc/motd. When a user successfully logs in, the contents of the /etc/motd file will be displayed as the message of the day.
chef cookbook generate created a recipes/default.rb file for you. By convention, this is the default location for your Chef code. All recipe .rb files containing Chef code are expected to be in the recipes/ subdirectory of a cookbook.
At the moment, recipes/default.rb doesn’t have very much in it, just some comments:
#
# Cookbook Name:: motd
# Recipe:: default
#
# Copyright (C) 2014
#
#
#
Add some Chef code to change the /etc/motd on your node by editing recipes/default.rb to resemble Example 7-3(we’ll leave it as an exercise to the reader to modify the copyright text).
Example 7-3. chefdk/motd/recipes/default.rb
#
# Cookbook Name:: motd
# Recipe:: default
#
# Copyright (C) 2014
#
#
#

cookbook_file "/etc/motd" do
  source "motd"
  mode "0644"
end
Here’s an explanation of what each line of code in Example 7-3 does:
  • cookbook_file is a Chef resource. The cookbook_file resource is used to transfer files from the files/subdirectory in a cookbook to the node.
  • do/end clauses note that the Chef resource definition spans multiple lines.
  • The "/etc/motd" string passed to cookbook_file is the namename that defines the path the file should be copied to on the node.
  • source defines the name of the file in the files/ subdirectory.
  • mode defines the octal permissions to set on the file after it is copied. In this case it is octal 644, “world readable.” If you don’t set the file mode appropriately, other users might not be able to read the contents of this file.
Now that you have created a cookbook directory structure using chef generate cookbook, skip ahead to Performing Your First Converge to use Chef to configure your node.

Your First Cookbook: Message of the Day (Chef Client)

You’ll be using a tool called knife to generate an initial cookbook directory structure for the message of the day cookbook (motd). knife is a basic utility command for working with Chef that you installed with Chef Client. On a command line, run the knife cookbook create subcommand to create the cookbook directory scaffolding. knife will create a main directory for your cookbook called motd as part of the process:
$ knife cookbook create motd --cookbook-path .
WARNING: No knife configuration file found
** Creating cookbook motd
** Creating README for cookbook: motd
** Creating CHANGELOG for cookbook: motd
** Creating metadata for cookbook: motd
Next, overlay all the files needed for your cookbook to enable Test Kitchen support, just like you did in Chapter 5:
$ cd motd
$ kitchen init --create-gemfile
      create  .kitchen.yml
      create  test/integration/default
      create  Gemfile
      append  Gemfile
      append  Gemfile
You must run `bundle install' to fetch any new gems.
Run bundle install to handle the extra Ruby dependencies:
$ bundle install
Fetching gem metadata from https://rubygems.org/..........
Resolving dependencies...
Using mixlib-shellout (1.4.0)
Using net-ssh (2.9.1)
Using net-scp (1.2.1)
Using safe_yaml (1.0.3)
Using thor (0.19.1)
Using test-kitchen (1.2.1)
Installing kitchen-vagrant (0.15.0)
Using bundler (1.5.3)
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.
Modify the .kitchen.yml file to use the CentOS 6 image we’ve tailored for the book as seen in Example 7-4.
Example 7-4. knife/motd/.kitchen.yml
---
driver:
  name: vagrant

provisioner:
  name: chef_solo

platforms:
  - name: centos65
    driver:
      box: learningchef/centos65
      box_url: learningchef/centos65

suites:
  - name: default
    run_list:
      - recipe[motd::default]
    attributes:
Check to make sure there are no syntax errors in your kitchen.yml file by running kitchen list. If you see a stack trace error instead of the following output, you likely made a typo, inadvertently used tabs instead of spaces, or didn’t line up entries correctly:
$ kitchen list
Instance          Driver   Provisioner  Last Action
default-centos65  Vagrant  ChefSolo     <Not Created>
Because you will be copying a file from the cookbook files/ subdirectory to the Chef node, you will need to create the file as well. Again, with your handy programmer’s text editor, create the file files/default/motd in your motd-knife cookbook. When a cow udders your message of the day (pun intended), any day becomes a celebration. Liven up your message with some ASCII art.
Example 7-5. knife/motd/files/default/motd
  _________________________________________________
  < YOU ARE ON A SIMULATED CHEF NODE ENVIRONMENT! >
   -----------------------------------------------
          \   ^__^
           \  (oo)\_______
              (__))\/\    \
                  ||----w |
                  ||     ||

Introducing the Cookbook_file Resource

We’ll use Chef to help us more easily determine that we are running on a guest virtual machine node by writing some automation to change the Linux message of the day. On Linux, the message of the day is displayed when a user logs in. The message of the day is used by Linux administrators to communicate with users. You can change the message of the day by editing the file /etc/motd. When a user successfully logs in, the contents of the /etc/motd file will be displayed as the message of the day.
knife cookbook create generates a recipes/default.rb file for you. By convention, this is the default location for your Chef code. All recipe .rb files containing Chef code are expected to be in the recipes/ subdirectory of a cookbook.
#
# Cookbook Name:: motd
# Recipe:: default
#
# Copyright 2014, YOUR_COMPANY_NAME
#
# All rights reserved - Do Not Redistribute
#
Add some Chef code to change the /etc/motd on your node by editing recipes/default.rb to resemble Example 7-6(we’ll leave it as an exercise to the reader to modify the copyright and licensing text in the comments later).
Example 7-6. knife/motd/recipes/default.rb
#
# Cookbook Name:: motd
# Recipe:: default
#
# Copyright 2014, YOUR_COMPANY_NAME
#
# All rights reserved - Do Not Redistribute
#

cookbook_file "/etc/motd" do
  source "motd"
  mode "0644"
end
Here’s an explanation of what each line in Example 7-6 does:
  • cookbook_file is a Chef resource. The cookbook_file resource is used to transfer files from the files/subdirectory in a cookbook to the node.
  • do/end clauses note that the Chef resource definition spans multiple lines.
  • The "/etc/motd" string passed to cookbook_file is the namename defines the path the file should be copied to on the node.
  • source defines the name of the file in the files/ subdirectory.
  • mode defines the octal permissions to set on the file after it is copied. In this case it is octal 644, “world readable.” If you don’t set the file mode appropriately, other users might not be able to read the contents of this file.
Now that you have created a cookbook directory structure using knife cookbook create, continue on to Performing Your First Converge to use Chef to configure your node. This is where the chef generate cookbook and knife cookbook create instructions in this chapter come together. The instructions that follow are the same for both tools.

Performing Your First Converge

Chef uses the term converge to refer to the process of deploying a cookbook to a node, running chef_client on the node, and applying a run list to put the node into a desired state. This is also referred to as converging a node. Let’s use Test Kitchen to perform a converge on the node using the kitchen converge command.

TIP

Make sure you run the kitchen converge command inside the motd cookbook directory.
If you entered in the code correctly so far, the output of kitchen converge should resemble the following:
$ kitchen converge default-centos65
-----> Starting Kitchen (v1.2.2.dev)
-----> Creating <default-centos65>...
       Bringing machine 'default' up with 'virtualbox' provider...
...
-----> Converging <default-centos65>...
       Preparing files for transfer
       Resolving cookbook dependencies with Berkshelf 3.1.3...
       Removing non-cookbook files before transfer
-----> Installing Chef Omnibus (true)
...
       Starting Chef Client, version 11.14.2
       [2014-08-14T13:22:37-07:00] INFO: *** Chef 11.14.2 ***
       [2014-08-14T13:22:37-07:00] INFO: Chef-client pid: 2004
       Creating a new client identity for default-centos65 using the validator
       key.
       [2014-08-14T13:22:40-07:00] INFO: Client key /tmp/kitchen/client.pem is
       not present - registering
       [2014-08-14T13:22:40-07:00] INFO: HTTP Request Returned 404 Not Found
       : Object not found: http://localhost:8889/nodes/default-centos65
       [2014-08-14T13:22:40-07:00] INFO: Setting the run_list to ["recipe
       [motd::default]"] from CLI options
       [2014-08-14T13:22:40-07:00] INFO: Run List is [recipe[motd::default]]
       [2014-08-14T13:22:40-07:00] INFO: Run List expands to [motd::default]
       [2014-08-14T13:22:40-07:00] INFO: Starting Chef Run for default-centos65
       [2014-08-14T13:22:40-07:00] INFO: Running start handlers
       [2014-08-14T13:22:40-07:00] INFO: Start handlers complete.
       [2014-08-14T13:22:40-07:00] INFO: HTTP Request Returned 404 Not Found :
       Object not found: /reports/nodes/default-centos65/runs
       resolving cookbooks for run list: ["motd::default"]
       [2014-08-14T13:22:40-07:00] INFO: Loading cookbooks [motd@0.1.0]
       Synchronizing Cookbooks:
       [2014-08-14T13:22:40-07:00] INFO: Storing updated cookbooks/motd
       /recipes/default.rb in the cache.
       [2014-08-14T13:22:40-07:00] INFO: Storing updated cookbooks/motd/
       README.md in the cache.
       [2014-08-14T13:22:40-07:00] INFO: Storing updated cookbooks/motd/
       metadata.json in the cache.
         - motd
       Compiling Cookbooks...
       Converging 1 resources
       Recipe: motd::default
         * cookbook_file[/etc/motd] action create[2014-08-14T13:22:40-07:00]
         INFO: Processing cookbook_file[/etc/motd] action create (motd::default
         line 10)
       [2014-08-14T13:22:40-07:00] INFO: cookbook_file[/etc/motd] backed up to
       /tmp/kitchen/backup/etc/motd.chef-20140814132240.562727
       [2014-08-14T13:22:40-07:00] INFO: cookbook_file[/etc/motd] updated file
       contents /etc/motd

           - update content in file /etc/motd from a7620c to 07a3b1
           --- /etc/motd        2014-07-04 07:47:33.211269359 -0700
           +++ /tmp/.motd20140814-2004-ky1c1j   2014-08-14 13:22:40.561072174
           -0700
           @@ -1,2 +1,9 @@
           -Welcome to your Packer-built virtual machine.
           +  _________________________________________________
           +  < YOU ARE ON A SIMULATED CHEF NODE ENVIRONMENT! >
           +   -----------------------------------------------
           +          \   ^__^
           +           \  (oo)\_______
           +              (__))\/\    \
           +                  ||----w |
           +                  ||     ||


       [2014-08-14T13:22:40-07:00] INFO: Chef Run complete in 0.117514782 seconds

       Running handlers:
       [2014-08-14T13:22:40-07:00] INFO: Running report handlers
       Running handlers complete
       [2014-08-14T13:22:40-07:00] INFO: Report handlers complete
       Chef Client finished, 1/1 resources updated in 2.729471192 seconds
       Finished converging <default-centos65> (0m28.88s).
-----> Kitchen is finished. (1m4.31s)

NOTE

As of this writing, there is a bug in the current Chef Development Kit 0.2.0 on Windows when you run kitchen converge. You’ll get an “SSL certificate verify failed” because the Chef Development Kit installation is not pointing at the correct certificate file. This issue is being tracked here: https://github.com/opscode/chef-dk/issues/106.
As a workaround, set the %SSL_CERT_FILE% environment variable before running kitchen converge:
Windows command prompt:
> set SSL_CERT_FILE=C:\opscode\chefdk\embedded\ssl\certs\cacert.pem
> kitchen converge
Windows PowerShell:
PS> Set-Item -Path env:SSL_CERT_FILE -Value \
C:\opscode\chefdk\embedded\ssl\certs\cacert.pem
PS> kitchen converge
Our output shows that Test Kitchen ran chef-client on the node. It reported Chef Run complete, how many resources were updated, and that Kitchen is finished with no errors.

NOTE

kitchen converge will automatically run kitchen create and kitchen setup for you if they are needed. In the preceding output, notice that the virtual machine was created, and chef-client was installed on the node, indicating that Test Kitchen automatically ran these two steps.

Validate Your Results

Use kitchen login to verify that your new message of the day is installed on the node:
$ kitchen login default-centos65
Last login: Thu Aug 14 13:22:36 2014 from 10.0.2.2
  _________________________________________________
  < YOU ARE ON A SIMULATED CHEF NODE ENVIRONMENT! >
   -----------------------------------------------
          \   ^__^
           \  (oo)\_______
              (__))\/\    \
                  ||----w |
                  ||     ||
Now we hope things are more clear when you are logged in to your simulated Chef node environment.

NOTE

The kitchen login command requires the ssh program to be installed on your host. On Windows, sshis not installed by default. If you get an error with the text “No such file or directory -ssh” when running kitchen login, make sure you have ssh installed, according to Install Unix Tools for Windows.
We’re done with the sandbox environment we created to verify the motd cookbook. Run the exit command on the guest to restore your command prompt back to the host:
[vagrant@default-centos65 ~]$ exit
logout
Connection to 127.0.0.1 closed.
Now run the kitchen destroy command to shut down the virtual machine and release all the associated system resources:
$ kitchen destroy default-centos65
-----> Starting Kitchen (v1.2.2.dev)
-----> Destroying <default-centos65>...
       ==> default: Forcing shutdown of VM...
       ==> default: Destroying VM and associated drives...
       Vagrant instance <default-centos65> destroyed.
       Finished destroying <default-centos65> (0m2.91s).
-----> Kitchen is finished. (0m3.37s)

NOTE

We will be creating a lot of different sandbox environments in this book. Don’t forget to kitchen destroy your environments when they are done so they won’t take up memory and disk space when you aren’t using them. (But just in case you forget to run kitchen destroy, we’ll keep reminding you.)
If you’d like to get a global overview of all the sandbox environments running on your machine from the command line, run vagrant global-status.

Anatomy of a Chef Run

kitchen converge performs a Chef run on your test node from your host. Pretty convenient! You can still use kitchen login to ssh into the node and poke around if you like, but the kitchen converge command is designed to give you fast feedback as you develop your cookbook. We’ll rely on kitchen converge for the rest of the hands-on exercises in this book.
In production, chef-client is typically run in daemonized mode as a service on the node, performing Chef runs at regular intervals; for example, once every 15 minutes. It checks in with Chef Server for any changes to cookbooks or the list of recipes to run on the node, which are stored on Chef Server. We’ll discuss this more in Chapter 9.
It is helpful to understand the steps involved in executing a Chef run. We’ve touched on some of them in Chapter 6, when we mentioned the node object and ohai, but we’ve yet to go over the steps explicitly, as shown in Figure 7-1.
  1. Start the Chef Client
    The chef-client process starts on the remote node. The process may be started by a service, cron job, or manually triggered by a user. The chef-client is the process responsible for evaluating Chef cookbooks containing recipes with Chef code on the target node.
  2. Build the node
    The chef-client process constructs the node object in memory. It runs ohai and gathers all the node’s automatic attributes (such as the hostname, fqdn, platform, users, etc.)
  3. Synchronize
    A run list is sent to the node. The run list contains a list of recipes to execute on the target node. A run list is the ordered, decomposed list of recipes to execute on the target node. The node may also be sent a list of URLs of cookbooks to download that are required by the run list. The target node will download and cache the required cookbooks in a local file cache.
  4. Load
    The cookbooks and Ruby components are loaded in this step. Cookbook-level attributes are merged with the automatic attributes generated by ohai in #2. The various components of a cookbook are loaded in this order:
    1. Libraries. All files in the libraries/ folder from every cookbook are loaded so that any language extensions or alterations are available for the remainder of the Chef run.
    2. Attributes. All files in the attributes/ folder from every cookbook are loaded and merged with the automatic ohai attributes.
    3. Definitions. All files in the definitions/ folder from every cookbook are loaded because definitions create resources and must be loaded before recipes.
    4. Resources. All files in the resources/ folder from every cookbook are loaded because resources must be loaded before the recipes.
    5. Providers. All files in the providers/ folder from every cookbook are loaded so the resources reference the proper provider.
    6. Recipes. All files in the recipes/ folder from every cookbook are loaded and evaluated. At this point, the recipes are not executed to place the node in the desired configuration, but the Ruby code is executed and each resource is added to the resource collection.
  5. Converge
    The converge phase is the most critical phase of a Chef run. This is when the Chef recipes are executed on the target node—packages are installed, templates are written, files are copied, and so on.
  6. Report
    If the Chef Client run is performed successfully, any new values in the node object are saved; otherwise, an exception is raised without updating the node object. Then notification and exception handlers are executed. Notification and exception handlers can perform a variety of functions, such as sending emails, posting to IRC, or sending messages to PagerDuty.
The run list is a key component used in a Chef run. As mentioned earlier, the run list contains a list of recipes to execute on the target node. It is not very common to pass a list of recipe .rb files, as we’ve done so far when running chef-client. Real-world chef runs typically involve dozens of cookbooks with possibly hundreds of recipes and associated files. There needs to be a succinct way of referring to all the files in a cookbook. That’s the purpose of a run list.
A run list is used to specify the cookbook recipes to be evaluated on a node. A run list specifies recipes in the form recipe['<cookbook_name>::<recipe_name>']; for example, recipe['motd::default']. When the Chef code is contained in the recipes/default.rb file of a cookbook, the recipe name is optional as the default is implied. recipe['motd'] is equivalent to recipe['motd::default']. Note that the .rb file extension is omitted when referring to a recipe, as this is assumed.
Anatomy of a Chef run
Figure 7-1. Anatomy of a Chef run
In the case of Test Kitchen, the run list is specified in the .kitchen.yml file and passed to chef-client on the command line via the -o parameter. In production, the run list is maintained on Chef Server as a node attribute.
We’ve just described how a Chef run works in more detail when you run kitchen converge. Now, let’s also cover the structure of a cookbook in more detail.

Cookbook Structure

A Chef cookbook is expected to have a specific file and directory structure. You could create these files and directories by hand with your editor and filesystem, but most people find it convenient to use the scaffolding generators Chef provides to lay down the directory structure: chef generate cookbook or knife cookbook create.
A basic cookbook directory structure contains the following files:
cookbook
├── .kitchen.yml
├── README.md
├── attributes
│    └── default/
├── chefignore
├── files/
│    └── default/
├── metadata.rb
├── recipes/
│    └── default.rb
└── templates/
    └── default/
\.kitchen.yml
The .kitchen.yml file is a YAML-format configuration file for Test Kitchen. You can use Test Kitchen to create sandbox environments to verify and validate your cookbook while you develop it.
README.md
Every cookbook should come with documentation. The README.md is a text file that contains documentation in markdown format. Markdown is a simple way of adding formatting to plain text so that you have the option of converting it to HTML. Markdown is a popular format for README files because the text files are readable as is, without the clutter of HTML tags and formatting instructions. Many popular source control tools will render README.md in HTML when viewing source, including GitHubGitLibStash, and Bitbucket.
attributes
You can provide your own custom attributes in a cookbook to complement (or override) the attributes generated automatically by ohai. Attributes are commonly used to define application distribution paths, platform-specific values, or software versions to install on a given node. The attributes/ directory can contain multiple .rb files with attribute definitions in them. When there is more than one attribute file, they are evaluated in alphabetical order. By convention, if it makes sense to store your attributes in one file, the file should be named default.rb.
chefignore
This file contains a list of files that should be ignored when uploading the cookbook to a Chef Server, when a Chef server is being used. By default, all files in a cookbook directory are uploaded to the Chef Server. There is no need, however, to upload things such as editor swapfiles or source control tracking files to a Chef Server. References to these files are commonly placed in a chefignore file so they won’t be uploaded.
files
The files folder is a centralized store in the cookbook for files to be distributed to nodes. Files can be plain text, images, zip files, and so on. These files can be deployed to a target node using the cookbook_fileresource. A directory structure underneath files/ controls whether files are copied to particular nodes. For more information on this structure, refer to http://docs.opscode.com/essentials_cookbook_files.html. When files should be distributed to all nodes, they are expected to be in the files/default/ subdirectory.
metadata.rb
The metadata.rb file contains all the information (metadata) about a cookbook. Every cookbook must have a metadata.rb file containing the name of the cookbook, version, dependencies, and other helpful information.
recipes
The recipes folder contains Chef recipes. A recipe file contains Chef code. There can be multiple .rb recipe files in the recipes/ folder. By convention, the main recipe file is called default.rb. For each node, Chef evaluates only recipe files that are specified on the node’s run list. The run list is specified in the .kitchen.ymlfile or on the chef-client command line, or stored on a Chef Server.
templates
The templates directory holds Chef templates. The templates directory is similar to files in that it contains a set of files to be distributed to nodes. However, the files used are Embedded Ruby templates. A template is plain text that can contain Ruby code, which is evaluated by the Embedded Ruby template engine before being rendered on a node. Templates are useful when you want to generate files with variable or selective content. templates follows the same directory-naming structure scheme as files to control whether the generated template files are copied to particular nodes.
We will cover all these structures in the next cookbook we write in this chapter. There are several more components to a Chef cookbook structure that are used by more advanced Chef cookbook coders. The additional components are Berksfiledefinitionslibrariesproviders, and resource. We won’t be covering these components as this book is intended for beginners, and all but the Berksfile are components used to customize Chef. Beginners usually need to spend time using Chef and discovering the limits of the built-in components before trying to tackle customization. Jon Cowie’s Customizing Chef book is the perfect resource when you are ready to tackle Chef customization.

NOTE

We really wanted to cover the Berksfile and its accompanying Berkshelf tool in this book, but the current 2.0 and 3.0 releases of Berkshelf are far too difficult for Ruby beginners to install using the Chef Client. Beginners basically must use the Chef Development Kit to install the Berksfile tool, but as we’ve already discussed, the Chef Development Kit isn’t available on all platforms at the time of this writing.
For more information on Berkshelf, refer to http://berkshelf.com and Jamie Winsor’s blog post on The Environment Cookbook Pattern. Jamie Winsor is the creator of the Berkshelf tool.

The Four Resources You Need to Know

If the full list of resources seems daunting, don’t worry: there are really only four types of resources you’ll find yourself using over and over:
package
Installs a package using the appropriate platform-native installer/package manager (yumaptpacman, etc.).
service
Manages the lifecycle of any daemons/services installed by the package resource.
cookbook_file
Transfers a file from the cookbook repository to a path on the node. We introduced the cookbook_fileresource earlier in this chapter to manage the /etc/motd file on our node.
template
A variant of the cookbook_file resource that lets you create file content from variables using an Embedded Ruby (ERB) template.
That’s it, just four resources. You’ll find yourself using these resources over and over again to install and configure apps and services. Let’s make this idea more concrete by writing one more cookbook to close out this chapter, a cookbook that will configure a web server to host our home page.

Apache Cookbook: A Step-By-Step Primer for Creating a Cookbook

We’ve formally introduced the steps in a Chef run and all the components of a cookbook directory structure. Now we will walk you through the step-by-step process we recommend to create cookbooks.
Before creating a cookbook, it’s important to define the purpose and scope of the cookbook. This step ensures each cookbook is truly a unit of your infrastructure. It also defines the vision and organization for the cookbook. A cookbook without a clear vision will likely become a source of difficulty in the future.
Completing Define Prerequisites might help focus your thoughts when you are creating a new cookbook. As you gain more experience writing Chef recipes, the thought process involved in developing a plan for your cookbook will become second nature without the need to formalize the plan in a checklist.

DEFINE PREREQUISITES

Table 7-1. Cookbook authoring checklist
Name
Purpose
Success criteria
App/Service
Required steps
Name
Although it might seem trivial, choosing a proper and descriptive name for a cookbook is absolutely critical. Because a cookbook’s name must be unique across your organization, you only have one chance to name it properly. For example, you can only have one “mysql” cookbook in your organization. A cookbook’s name is also an abbreviated statement of work and should follow the principle of least surprise. For example, the “mysql” cookbook should deal exclusively with MySQL.
Purpose
The purpose or goal of a cookbook is the second-most important prerequisite when creating a cookbook. A cookbook without a proper vision is certain to fail, if not immediately, then in the long term due to scope creep resulting in untestable code. The following is a vision statement for our mysql cookbook:
To install and configure the MySQL server and MySQL client on a target machine.
The vision is sometimes closely tied to the metadata’s description attribute, but this is not a requirement.
Success criteria
Once you have a goal, you need to have some way to determine that you have achieved the goal. Success criteria describe the things we need to evaluate about the cookbook to determine that it meets its intended purpose. The following is an example of success criteria for the mysql cookbook:
Do just enough to get MySQL running and expose a way to create MySQL users, databases, and tables.
App/service
Each cookbook should manage a single application or service (a unit of your infrastructure). You should identify that application or service before creating the cookbook. If you are unable to identify a single application or service, you should consider narrowing the vision from the previous step. For example, our narrowed “App/Service” would be “MySQL.”
Required steps
If you don’t know the required steps to accomplish your goal, it’s going to be difficult to automate the process with a cookbook. Automation first requires a grasp of what steps are needed to install and configure the application manually (along with any prerequisites), then you can start tackling the process of making the installation repeatable without human intervention via automation.
In this section, we will create and author a cookbook named “apache” to configure our web server. Table 7-2 is the completed table for our “apache” cookbook.
Table 7-2. Apache cookbook checklist
Name
apache
Purpose
To configure a web server to serve up our home page.
Success Criteria
We can see the home page in a web browser.
App/Service
Apache HTTP server
Required Steps
1) Install apache, 2) Start the service, 3) Configure service to start when machine boots, 4) Write out the home page
With this cookbook, our goal is to teach you the three remaining Chef resource primitives not yet covered and to outline the steps involved in creating a cookbook, not web server management. These are not the exact steps required to configure the Apache HTTP server configuration in production, but they’re perfectly fine for this example. Configuring a real-world Apache HTTP server would involve more configuration steps, but it would still involve these four basic resources.
Now that we have a clear vision for the apache cookbook, we can generate the cookbook skeleton.

GENERATE THE COOKBOOK SKELETON

We’ve already been through the steps to generate the scaffolding for a cookbook using chef generate cookbook or knife cookbook create, depending on whether you have the Chef Development Kit or Chef Client installed on your development workstation. In this case, you’ll want to create a cookbook named apache.
First, you need to create the cookbook skeleton with Test Kitchen support. We’re going to go quickly through the cookbook creation commands this time without showing you the tool output. Refer back to Your First Cookbook: Message of the Day (Chef Development Kit) if you’re using the Chef Development Kit, or Your First Cookbook: Message of the Day (Chef Client) if you are using Chef Client for a refresher on the steps involved.
Chef Development Kit:
$ chef generate cookbook apache
$ cd apache
Chef Client:
$ knife cookbook create apache --cookbook-path .
$ cd apache
$ kitchen init --create-gemfile
$ bundle install
Edit the .kitchen.yml to use CentOS image tailored for use in this book.

NOTE

From this point forward, we’ll provide listings only for the Chef Development Kit versions of the source files, and not Chef Client. There are no functional differences between the two versions. You’ll just notice that knife creates a few more file types by default that aren’t covered in this book, and some of the comments are different. Having the extra generated files doesn’t hurt anything as they all are just blank stubs we won’t be using.
Example 7-7. chefdk/apache/.kitchen.yml
---
driver:
  name: vagrant

provisioner:
  name: chef_zero

platforms:
  - name: centos65
    driver:
      box: learningchef/centos65
      box_url: learningchef/centos65

suites:
  - name: default
    run_list:
      - recipe[apache::default]
    attributes:
Example 7-7 shows how the run list is defined in the suites: stanza of the .kitchen.yml. Chef Development Kit users, remember to make sure this is correct, due to the current bug that incorrectly sets the run list to recipe[bar::default].
Now you know that the run list recipe[apache::default] is just shorthand indicating that Chef code needs to be executed in the recipes/default.rb file of the apache cookbook. Chef-client executes the code when we run kitchen converge.
To make sure your .kitchen.yml has no syntax errors, run kitchen list. If there are issues, correct them:
$ kitchen list
Instance          Driver   Provisioner  Last Action
default-centos65  Vagrant  ChefZero     <Not Created>

EDIT THE README.MD FILE

Your README.md file should drive the cookbook development, based on the prerequisites you defined in Define PrerequisitesExample 7-8 shows a suggested README.md for your apache cookbook.
Example 7-8. chefdk/apache/README.md
# apache cookbook

This cookbook installs and configures a simple web site using the Apache HTTPD server.

Requirements
============
Supports only CentOS or other RHEL variants that use the +httpd+ package.

Usage
=====
Add `apache` to your node's `run_list`.

Testing
=======
A `.kitchen.yml` file is provided.  Run +kitchen converge+ to verify this cookbook.

UPDATE METADATA.RB

Here’s the metadata.rb file that was generated in your cookbook skeleton:
name             'apache'
maintainer       ''
maintainer_email ''
license          ''
description      'Installs/Configures apache'
long_description 'Installs/Configures apache'
version          '0.1.0'
Note the apache string in the name field. This is how Chef determines the name of your cookbook in the run list. It does not look at the name of the directory in which the cookbook files reside. For your own sanity, the namefield in the metadata.rb and your cookbook directory should match, but they aren’t required to.
Edit the metadata.rb file, adding your name and e-mail address in the maintainer and maintainer_emailfields. Because you will want to share your cookbooks, it is also a good idea to indicate how you intend to make your code sharable by listing one of the standard open source license types. Refer to the site http://choosealicense.com for more information on open source software licensing.
Example 7-9 shows how I filled out my metadata.rb file. When you edit the metadata.rb file, use values that are appropriate for you.
Example 7-9. chefdk/apache/metadata.rb
name             'apache'
maintainer       'Mischa Taylor'
maintainer_email 'mischa@misheska.com'
license          'MIT'
description      'Installs/Configures apache'
long_description 'Installs/Configures apache'
version          '0.1.0'

INTRODUCING THE PACKAGE RESOURCE

We haven’t written any Chef code in our recipe yet, but go ahead and run an initial kitchen converge on your cookbook skeleton to make sure there aren’t any syntax errors in the files you have edited so far. We encourage you to run kitchen converge frequently to verify your cookbook code as you write it. The first kitchen converge will take a few minutes, as Test Kitchen needs to set up the sandbox environment and it will also download and install chef-client on the node. But subsequent kitchen converge runs will go pretty quickly. Run kitchen converge now:
$ kitchen converge default-centos65
Now, let’s get to some coding. First, let’s use the package resource to install the httpd package using yum install by editing recipes/default.rb, as shown in Example 7-10.
Example 7-10. chefdk/apache/recipes/default.rb
#
# Cookbook Name:: apache
# Recipe:: default
#
# Copyright (C) 2014
#
#
#

package "httpd" do
  action :install
end
Run kitchen converge again to check your work. We also encourage you to use kitchen login to inspect the node to verify that your recipe is doing what you expect. Log in to the node, and verify that the httpd service is installed with the following command. Be sure to exit back out to your host prompt when you are done:
$ kitchen converge default-centos65
$ kitchen login default-centos65
Last login: Thu Aug 14 13:48:32 2014 from 10.0.2.2
Welcome to your Packer-built virtual machine.
[vagrant@default-centos65 ~]$ rpm -q httpd
httpd-2.2.15-31.el6.centos.x86_64
[vagrant@default-centos65 ~]$ exit
logout
Connection to 127.0.0.1 closed.
The httpd service is indeed installed! The rpm -q command queries the rpm package manager database to see if a package containing the name is present on the system.
If you look at the Chef documentation on the package resourcepackage calls one of many providers based on the platform and platform_family returned by ohai. Because our platform is rhel, theyum_package provider is used.
If you refer to the documentation on the yum_package provider, you’ll notice that there are four possible actions: :install:upgrade:remove, and :purge. Because :install is the default, we don’t need to specify the action. Let’s change our recipe accordingly, as shown in Example 7-11.
Example 7-11. chefdk/apache/recipes/default.rb
#
# Cookbook Name:: apache
# Recipe:: default
#
# Copyright (C) 2014
#
#
#

package "httpd"
Now the package reference is a lot more concise, and it performs the same action. Look for opportunities to make use of default actions where possible.

INTRODUCING THE SERVICE RESOURCE

Next, let’s use the service resource to start the httpd service and automatically enable it on restart. The service resource can start a service with the :start action, and it can enable a service at boot with the :enable action.
You can pass more than one action to the service resource by passing them as an array. In Chapter 3 we discussed how an array is a delimited list of comma-delimited items contained within square brackets ([]).
Add the service resource to recipes/default.rb as shown in Example 7-12.
Example 7-12. chefdk/apache/recipes/default.rb
#
# Cookbook Name:: apache
# Recipe:: default
#
# Copyright (C) 2014
#
#
#

package "httpd"

service "httpd" do
  action [ :enable, :start ]
end
This is the first time we’ve encountered multiple resources in one recipe. Chef evaluates the recipes as you would expect—in the order they are listed in the file.
Let’s run kitchen converge again and log in to verify that our cookbook produced the intended result:
$ kitchen converge default-centos65
$ kitchen login default-centos65
Last login: Thu Aug 14 13:50:39 2014 from 10.0.2.2
Welcome to your Packer-built virtual machine.
[vagrant@default-centos65 ~]$ chkconfig --list httpd | grep 3:on
httpd              0:off        1:off   2:on    3:on    4:on    5:on    6:off
[vagrant@default-centos65 ~]$ exit
logout
Connection to 127.0.0.1 closed.
Our service is enabled! For CentOS and other Redhat variants, services are enabled in different run levels. Our service should be enabled for runlevel 3, which is multi-user text mode with networking enabled—the default for a working CentOS server running in text mode. In our grep statement, we verified that the service is set to be onfor runlevel 3.

INTRODUCING THE TEMPLATE RESOURCE

We’ll introduce the template resource by showing you how to generate the file containing the content for our website. The template resource is similar to cookbook_file in that it creates a file on the node. However, a template has the additional ability to expand variable references to the file and other statements in the form ofEmbedded RuBy (ERB).
Let’s add the resource statement to recipes/default.rb as shown in Example 7-13.
Example 7-13. chefdk/apache/recipes/default.rb
#
# Cookbook Name:: apache
# Recipe:: default
#
# Copyright (C) 2014
#
#
#

package "httpd"

service "httpd" do
  action [ :enable, :start ]
end

template "/var/www/html/index.html" do
  source 'index.html.erb'
  mode '0644'
end
By default, the httpd server looks for web pages in the directory /var/www/html. The file for the default website is expected to be in the file /var/www/html/index.html. We also still have to set the file mode to be world-readable. What’s probably new to you, compared to the file resource, is this source attribute.
The source attribute specifies the file containing a template with ERB statements. This template is expected to be located underneath the templates folder in the cookbook, following the same subdirectory convention as files. So to copy the template to all nodes (which is the default), make sure that template files are located in the templates/default directory.
Let’s create an ERB template for our index.html file. By convention, an ERB template is expected to have the suffix .erb appended to the generated filename. The Chef Development kit does this expansion for you automatically when you run chef generate template index.html, creating the file templates/default/index.html.erb. With the Chef Client, you must create this file manually.
Chef Development Kit:
$ chef generate template index.html
Chef Client - Linux/Mac OS X:
$ touch templates/default/index.html.erb
Chef Client - Windows:
$ touch templates\default\index.html.erb
Variables get expanded in an ERB file when Chef sees statements bounded by <%= and %>, such as <%= node['hostname'] %> in the following index.html.erb file. Chef evaluates the node['hostname'] variable and replaces the contents of the <%= node['hostname'] %> block with the resultant value when the file is written out to the node. Edit templates/default/index.html.erb as shown in Example 7-14.
Example 7-14. chefdk/apache/templates/default/index.hmtl.erb
This site was set up by <%= node['hostname'] %>
Run kitchen converge, then run kitchen login to verify that our template resource created the file. Check to make sure the file exists at /var/www/html/index.html, and use the curl to see the web page rendered on the command line. Note that when the /var/www/html/index.html file was created, the <%= node['hostname'] %> string in our template was replaced by the value of the node['hostname']:
$ kitchen converge default-centos65
$ kitchen login default-centos65
Last login: Thu Aug 14 13:52:00 2014 from 10.0.2.2
Welcome to your Packer-built virtual machine.
[vagrant@default-centos65 ~]$ more /var/www/html/index.html
This site was set up by default-centos65
[vagrant@default-centos65 ~]$ curl localhost
This site was set up by default-centos65
[vagrant@default-centos65 ~]$ exit
logout
Connection to 127.0.0.1 closed.
While it is great to see this output, we haven’t quite yet met our defined success criteria for this cookbook. We need to be able to see the home page in a web browser on your host. Let’s do that in the next section.

VERIFY SUCCESS CRITERIA ARE MET

In order to give your host access to the website on your guest, you’ll need to assign a known, static IP to your node in your .kitchenl.yml. This is done by adding a driver: network: block to your .kitchen.yml in the following form:
driver:
  network:
  - ["private_network", {ip: "192.168.33.7"}]
The static IP address should be chosen from the TCP/IP reserved private address space that does not conflict with other machines on the same network. The IP address 192.168.33.7 should work for nearly everyone, as most routers don’t use this subnet by default, so modify your .kitchen.yml file accordingly, as shown in Example 7-15.
Example 7-15. chefdk/apache/.kitchen.yml
---
driver:
  name: vagrant

provisioner:
  name: chef_zero

platforms:
  - name: centos65
    driver:
      box: learningchef/centos65
      box_url: learningchef/centos65
      network:
      - ["private_network", {ip: "192.168.33.7"}]

suites:
  - name: default
    run_list:
      - recipe[apache::default]
    attributes:
Unfortunately, Test Kitchen will apply network configuration settings in the .kitchen.yml only when running kitchen create the first time, when the sandbox environment is created. Because we have already created the sandbox environment, we’ll need to run kitchen destroy first, before running kitchen converge, so it will create a new sandbox environment. Otherwise, Test Kitchen will ignore the networking change we just added to the kitchen.yml:
$ kitchen destroy default-centos65
$ kitchen converge default-centos65
Now that the sandbox environment has been created with an IP address accessible from our host, try it out in your web browser using the address http://192.168.33.7. You should see the template file we just created, as in Figure 7-2. Success criteria met!
Your apache site on 192.168.33.7
Figure 7-2. Your apache site on 192.168.33.7
If you get an error, check the following:
  1. Make sure you created the file apache/templates/default/index.html.erb and it has the correct content.
  2. Run kitchen login, then run curl localhost to make sure that Chef run completed properly and the web server is working within the virtual machine.
  3. Scrutinize the kitchen converge output to make sure there was no error when vagrant configured the private_network address when it set up the virtual network adapters on the virtual machine.
  4. There is no possibility that another machine on the local network has the same IP address. If so, modify the .kitchen.yml and recreate the virtual machine.
We are now done with this cookbook and virtual machine. Run the kitchen destroy command to shut down the virtual machine and release all the associated system resources:
$ kitchen destroy default-centos65
-----> Starting Kitchen (v1.2.2.dev)
-----> Destroying <default-centos65>...
       ==> default: Forcing shutdown of VM...
       ==> default: Destroying VM and associated drives...
       Vagrant instance <default-centos65> destroyed.
       Finished destroying <default-centos65> (0m2.95s).
-----> Kitchen is finished. (0m3.43s)

Summary

In this chapter, we introduced the concept of a cookbook. Chef needs more than recipe files with code to automate the configuration of nodes. A cookbook contains all the other associated information, packing everything together into a single unit of deployment. We covered three of these additional components of a cookbook in this chapter: the metadata.rb file, the files folder, and the templates folder. We also showed you that Chef code resides in the recipes folder.
We introduced you to the four essential resources you’ll find yourself using over and over again in your recipe code:
package
Installs a package using the system package manager
service
Manages the lifecycle of any daemons/services installed by the package resource
cookbook_file
Transfers a file from the files folder of a cookbook to a path on the node
template
A variant of the cookbook_file resource that lets you create file content from variables using an Embedded Ruby (ERB) template. Templates are located in the templates folder of a cookbook
Finally, we walked you through a process we recommend you use when creating a cookbook:
  1. Define prerequisites and goals.
  2. Generate the cookbook skeleton.
  3. Let the documentation you write in the README.md file guide development.
  4. Define metadata in the metadata.rb file.
  5. Verify cookbook code as you write it using kitchen converge and kitchen login.
  6. Verify conditions of success are met.
In the next chapter, we’ll delve a bit more deeply into attributes. You can create your own custom attributes. We’ll cover where and how attributes are set in more detail.

7 comments: