Chapter 11. Chef Zero

Wouldn’t it be nice to be able to spin up a Chef Server locally using a smaller memory footprint than shown in Chapter 9? On many systems, requiring 2 GB of free memory just to simulate a production Chef Server environment is a lot to ask. The Chef Development Kit and Chef Client just so happen to include a stripped-down version of Chef Server for this very purpose, called chef-zero.

chef-zero runs comfortably in as little as 20 MB of memory. Because it is small, it also starts up quickly, which is great for testing. In order to fit into such a small memory footprint, chef-zero sacrifices a few things. There is no web UI, nor is there any persistence; once Chef Zero is stopped, all data is lost. Neither of these two things is needed for testing.

Test Kitchen provides built-in support for chef-zero. Let’s go through a simple example of how you can use chef-zero with Test Kitchen. It’s great for testing your cookbook in a sandbox environment with chef-client using chef-zero as a simulated Chef Server, so you can test cookbooks that exploit Chef Server-specific features. We’ll be covering more of these server-specific features in the remainder of this book, so having a nimbler test environment available will be handy.

For those using Chef Client, you will need to install an additional gem to use Chef Zero. Run the following command to install chef-zero:

$ sudo gem install chef-zero --no-ri --no-rdoc

If you’re using the Chef Development Kit, you’re fine, this Ruby gem has already been installed for you.

Test Kitchen and Chef Zero

Generate a cookbook named zero with chef generate cookbook or knife cookbook create, depending on whether you are using the Chef Development Kit or the Chef Client respectively. Also, enable the cookbook to use Test Kitchen. We’re going to go through the cookbook creation steps quickly in this chapter. If you need a refresher on what each of these commands mean and the expected output, refer back to Chapter 7.

Chef Development Kit:

$ chef generate cookbook zero
$ cd zero

Chef Client:

$ knife cookbook create zero --cookbook-path .
$ cd zero
$ kitchen init --create-gemfile
$ bundle install

Edit the provisioner: stanza in the generated .kitchen.yml to use chef_zero. As of this writing, chef_zerois not the default provisioner, but it might be by the time you read this. Also, edit the .kitchen.yml file to use the CentOS 6.5 basebox we prepared specifically for this book. Next, assign a private network address like we did in Chapter 7. This time, we’re going to use the IP address 192.168.33.34. If this conflicts with an address already being used on your local network, change it to be a nonconflicting one.

Example 11-1. chefdk/zero/.kitchen.yml

---
driver:
  name: vagrant

provisioner:
  name: chef_zero

platforms:
  - name: centos65
    driver:
      box: learningchef/centos65
      box_url: learningchef/centos65

suites:
  - name: default
    run_list:
      - recipe[zero::default]
    attributes:

Because we just want to present an overview of what chef-zero does, we’re going to use the default-generated cookbook, which does nothing. Perform a kitchen converge to perform a Chef run using chef-zero. You should notice that the output looks a little different than when you used chef-solo:

$ kitchen converge
-----> Starting Kitchen (v1.2.2.dev)
-----> Creating <default-centos65>...
-----> Converging <default-centos65>...
-----> Installing Chef Omnibus (true)
       Transferring files to <default-centos65>
       [2014-07-22T10:31:35-07:00] INFO: Starting chef-zero on port 8889 with
       repository at repository at /tmp/kitchen
         One version per cookbook

       [2014-07-22T10:31:35-07:00] INFO: Forking chef instance to converge...
       Starting Chef Client, version 11.12.8
       [2014-07-22T10:31:35-07:00] INFO: *** Chef 11.12.8 ***
       [2014-07-22T10:31:35-07:00] INFO: Chef-client pid: 2004
       Creating a new client identity for default-centos65 using the validator
       key.
       [2014-07-22T10:31:37-07:00] INFO: Client key /tmp/kitchen/client.pem
       is not present - registering
       [2014-07-22T10:31:37-07:00] INFO: HTTP Request Returned 404 Not Found :
       Object not found: http://127.0.0.1:8889/nodes/default-centos65
       [2014-07-22T10:31:37-07:00] INFO: Setting the run_list to
       ["recipe[zero::default]"] from CLI options
       [2014-07-22T10:31:37-07:00] INFO: Run List is [recipe[zero::default]]
       [2014-07-22T10:31:37-07:00] INFO: Run List expands to [zero::default]
       [2014-07-22T10:31:37-07:00] INFO: Starting Chef Run for default-centos65
       [2014-07-22T10:31:37-07:00] INFO: Running start handlers
       [2014-07-22T10:31:37-07:00] INFO: Start handlers complete.
       [2014-07-22T10:31:38-07:00] INFO: HTTP Request Returned 404 Not Found :
       Object not found: /reports/nodes/default-centos65/runs
       resolving cookbooks for run list: ["zero::default"]
       [2014-07-22T10:31:38-07:00] INFO: Loading cookbooks [zero@0.1.0]
       Synchronizing Cookbooks:
       [2014-07-22T10:31:38-07:00] INFO: Storing updated
       cookbooks/zero/recipes/default.rb in the cache.
       [2014-07-22T10:31:38-07:00] INFO: Storing updated
       cookbooks/zero/README.md in the cache.
       [2014-07-22T10:31:38-07:00] INFO: Storing updated
       cookbooks/zero/metadata.json in the cache.
         - zero
       Compiling Cookbooks...
       Converging 0 resources
       [2014-07-22T10:31:38-07:00] INFO: Chef Run complete in 0.043313495 seconds

       Running handlers:
       [2014-07-22T10:31:38-07:00] INFO: Running report handlers
       Running handlers complete

       [2014-07-22T10:31:38-07:00] INFO: Report handlers complete
       Chef Client finished, 0/0 resources updated in 2.396534174 seconds
       Finished converging <default-centos65> (0m29.51s).
-----> Kitchen is finished. (1m4.76s)

Here’s an overview of the steps Test Kitchen performed to set up chef-zero in the sandbox environment. It:

1. Installed Chef Client
2. Created fake validation.pem and client.pem keys in /tmp/kitchen
3. Generated client.rb (the configuration file for chef-client) in /tmp/kitchen
4. Generated dna.json file with run list in /tmp/kitchen
5. Synchronized cookbooks on host in /tmp/kitchen/cookbooks
6. Ran chef-client in local mode. The full command line used is chef-client --local-mode --config /tmp/kitchen/client.rb --log_level --chef-zero-port 8889 --json-attributes dna.json

Destroy the sandbox environment; we’re done with it for now:

$ kitchen destroy

One important thing to remember is that Test Kitchen runs chef-zero in the background during the Chef run, then stops chef-zero once the Chef run is complete. It does not leave chef-zero running, nor does it configure knife to run in your sandbox environment. As discussed in Chapter 9, knife is the primary tool for interacting with a Chef Server. It is handy to be able to simulate knife as well in a test environment. But we need to do a few more things in order to also simulate a Chef Server using knife.

Running Chef-Zero on Your Host Using Chef-Playground

You can also run chef-zero on your host. The most likely reason you’ll want to do this is to simulate a Chef Server so you can run the knife tool, like we did in Chapter 9. When you want to interact with a Chef Server, you’ll find yourself using knife on your host Development Workstation even when you are using Test Kitchen. Also, some Chef Server features such as data bags or search really benefit from being able to use knife, even during testing.

We’ll be creating a project directory called chef-playground which models the chef-repo setup we used in Chapter 9, but uses chef-zero instead. We’ll follow similar steps that we used in Test Kitchen and Chef Zero:

1. Assume Chef Client or the Chef Development Kit is installed.
2. Create fake validation.pem and client.pem keys.
3. Create knife.rb (the configuration file for knife).
4. Run chef-zero.
5. Synchronize cookbooks with chef-zero simulated Chef Server.
6. Run knife.

Create a directory called chef-playground, and make it the current working directory:

$ mkdir chef-playground
$ cd chef-playground

Then create another subdirectory called .chef (similar to the chef-repo/.chef directory we created in Chapter 9), which will contain our fake keys and configuration files:

$ mkdir .chef
$ cd .chef

Use the ssh-keygen tool to generate some client keys. They don’t need to be real keys tied to a user or to Chef, but they do need to contain a readable key. We discussed the purpose of the client.pem file in Chapter 10. In this case, we’ll name the file devhost.pem, which matches the devhost name we’ll be using for our Development Workstation. Enter in the following command lines. (The -P option supplies a passphrase for the key. In this case, we don’t want a passphrase, so we pass in double quotes [""] to supply a blank password.)

Linux/Mac OS X/Windows Command Prompt:

$ ssh-keygen -f devhost.pem -P ""
$ ssh-keygen -f validation.pem -P ""

Windows PowerShell:

$ ssh-keygen --% -f devhost.pem -P ""
$ ssh-keygen --% -f validation.pem -P ""

Create a knife.rb file in the chef-playground/.chef directory as shown in Example 11-2. This is the final configuration file you need to create.

Example 11-2. chef-playground/.chef/knife.rb

chef_repo = File.join(File.dirname(__FILE__), "..")

chef_server_url "http://127.0.0.1:9501"
node_name       "devhost"
client_key      File.join(File.dirname(__FILE__), "devhost.pem")
cookbook_path   "#{chef_repo}/cookbooks"
cache_type      "BasicFile"
cache_options   :path => "#{chef_repo}/checksums"

Finally, open up a separate terminal window and run chef-zero alongside the command prompt in which you are doing these hands-on-exercises, as shown in Figure 11-1. Run chef-zero as shown in the following code, passing in a port number besides the default port 8889, so you won’t conflict with other Chef tools running on your host in local mode. If you discover a conflict with the suggested port 9501, use another.

$ chef-zero --port 9501

NOTE

You can run chef-zero in “daemonized” mode by passing in the --daemon parameter. Chef-zero will detach itself from the current command line process and run in the background.

As shown in Figure 11-1, when run on a command line, chef-zero will display that it is listening, not returning to the command prompt. This is why we recommend running it in a separate window. Leave chef-zero running for now.

Figure 11-1. Run chef-zero in a window alongside your cookbook

Make sure that the chef-playground directory is the current working directory. Assuming everything is configured properly, when you run knife client list, it should return chef-validator and chef-webui as shown in the following code block:

$ pwd
/Users/misheska/chef-playground

$ knife client list
chef-validator
chef-webui

The knife tool will look for configuration files and credentials in the $HOME/.chef directory by default. If knife doesn’t find anything in this default location, it will then walk up a directory tree looking for the first .chefdirectory, if it exists. This is a recommended way to arrange your configuration files, if you have to work against multiple Chef servers using the Chef tools in client mode—sprinkle .chef directories in root locations that make sense for the project, such as chef-playground/.chef.

NOTE

We won’t be using knife --local-mode in this book, but it’s helpful to mention. Similar to chef-client, the knife tool supports a local mode using the --local-mode option. A benefit to using local mode with knife is that it will automatically start chef-zero for you.

You could have run the following to run knife in local mode to check clients. A benefit to this approach is that it will automatically start chef-zero for you. This doesn’t conflict with the chef-zero instance you already started because it is running on a different port than the default port 8889.

$ knife client list --local-mode

However, you’ll notice that the output differs compared to when knife is running in “client” mode. So we won’t be making use of the local mode feature in this book. Jon Cowie’s Customizing Chef book covers the use of both chef-client and knife using local mode.

Before we finish with our chef-playground project, let’s pre-populate chef-zero with some node information so we can get more useful test results back from search queries against Chef Server.

Create a directory called nodes underneath chef-playground, and make it the current working directory:

$ mkdir nodes
$ cd nodes

Within the chef-playground/nodes directory, create three files, as shown in Example 11-3, Example 11-4, and Example 11-5. When you are done, the chef-playground directory structure should resemble the following:

chef-playground/
├── .chef
│   ├── devhost.pem
│   ├── devhost.pem.pub
│   ├── knife.rb
│   ├── validator.pem
│   └── validator.pem.pub
└── nodes
    ├── atwood.json
    ├── snowman.json
    └── susu.json

Example 11-3. chef-playground/nodes/atwood.json

{
  "name": "atwood",
  "chef_type": "node",
  "json_class": "Chef::Node",
  "chef_environment": "_default",
  "run_list": ["recipe[apache]", "recipe[motd]"],
  "automatic": {
    "ipaddress": "192.168.33.31",
    "hostname": "atwood",
    "fqdn": "atwood.playground.local",
    "os": "linux",
    "os_version": "2.6.32-431.el6.x86_64",
    "platform": "centos",
    "platform_version": "6.5",
    "platform_family": "rhel",
    "recipes": ["apache", "motd"]
  }
}

Example 11-4. chef-playground/nodes/snowman.json

{
  "name": "snowman",
  "chef_type": "node",
  "json_class": "Chef::Node",
  "chef_environment": "_default",
  "run_list": ["recipe[apache]", "recipe[motd]", "recipe[motd-attributes]"],
  "automatic": {
    "ipaddress": "192.168.33.32",
    "hostname": "snowman",
    "fqdn": "snowman.playground.local",
    "os": "linux",
    "os_version": "3.13.0-24-generic",
    "platform": "ubuntu",
    "platform_version": "14.04",
    "platform_family": "debian",
    "recipes": ["apache", "motd", "motd-attributes"]
  }
}

Example 11-5. chef-playground/nodes/susu.json

{
  "name": "susu",
  "chef_type": "node",
  "json_class": "Chef::Node",
  "chef_environment": "_default",
  "run_list": ["recipe[apache]", "recipe[motd]"],
  "automatic": {
    "ipaddress": "192.168.33.33",
    "hostname": "susu",
    "fqdn": "susu.playground.local",
    "os": "linux",
    "os_version": "2.6.32-431.el6.x86_64",
    "platform": "centos",
    "platform_version": "6.5",
    "platform_family": "rhel",
    "recipes": ["apache", "motd"]
  }
}

Once you have the files in the nodes/ subdirectory created, make sure that chef-playground is the current working directory. Then, run the knife upload command to create the node information on the server. We’ll use this knife upload technique in subsequent chapters of this book to pre-populate the chef-zero server with test data before running other knife commands:

$ pwd
/Users/misheska/chef-playground

$ knife upload nodes
Created nodes/atwood.json
Created nodes/snowman.json
Created nodes/susu.json

Now if you run the knife node list command, you’ll see that chef-zero thinks that there are three nodes being managed:

$ knife node list
atwood
snowman
susu

That’s a quick overview of Chef Zero. We’ll be using Chef Zero to show you more Chef Server functionality in upcoming chapters. Hit Ctrl-C in the window in which you launched chef-zero to stop the Chef Zero server.

Summary

In this chapter, we showed you how Chef Zero provides a complete, in-memory version of Chef Server that is easy to install and great for checking out features of Chef Server locally without needing to have a full Chef Server setup. We’ll be using this nimbler implementation of Chef Server for the rest of the exercises in this book.