Chapter 15. Environments

Environments are a feature of Chef Server used to model the server configurations required during each phase of your software development lifecycle, as shown in Figure 15-1.

Figure 15-1. Environments overview

Environments reflect your patterns and workflow, and can be used to model the life stages of your application, such as:

• Development
• Testing
• Staging
• Production

Every Chef Server starts out with a single environment, the _default environment.

Environments might include attributes necessary for configuring your infrastructure, such as:

• The URL of a payment service API
• The location of a package repository
• The version of Chef configuration files that should be used

Environments allow for isolating resources on a Chef Server because environments can contain version constraints, unlike with roles. Environments still have a use even when you have Test Kitchen at your disposal, because you’ll probably want to do some testing against servers in your production environment.

Create a Dev Environment

Environments can be created and managed in the same fashion as data bags and roles, organized in a directory under chef-playground. The directory name is environments by default.

Use the chef-playground directory you created in Chapter 11. Use the same dual command prompt setup you used there. Start the chef-zero server on an open port in one window. We will be using port 9501 in the examples in this chapter:

$ chef-zero --port 9501

Make sure that the chef-playground directory is the current working directory:

$ cd chef-playground

Create an environments directory in chef-playground:

$ mkdir environments

We’re going to create a .json file representing the new environment. A basic environment has a name: and description:. Environments can have one or more cookbook constraints as well. The ability to “pin” cookbooks to particular versions is the most useful feature of environments. Create the file chef-playground/roles/dev.json containing the code provided in Example 15-1.

Example 15-1. chef-playground/environments/dev.json

{
  "name": "dev",
  "description": "For developers!",
  "cookbook_versions": {
    "apache": "= 0.2.0"
  },
  "json_class": "Chef::Environment",
  "chef_type": "environment"
}

NOTE

There are other options for a version constraint besides “equal to” (=). Equality is the recommended practice. To learn more about the other options, refer to http://bit.ly/abt_cookbook_versions.

Run knife environment from file passing in the dev.json file. knife environment from fileassumes dev.json is located in a subdirectory called environments, and not in the current directory:

$ knife environment from file dev.json
Updated Environment dev

Run knife environment show dev as follows to display the details about the dev environment:

$ knife environment show dev
chef_type:           environment
cookbook_versions:
  apache: = 0.2.0
default_attributes:
description:         For developers!
json_class:          Chef::Environment
name:                dev
override_attributes:

Attributes and Environments

Environments can contain attributes. Let’s experiment with this by creating a .json file to represent a production environment. This environment will pin production to the older version of the apache cookbook, the 0.1.0 that is not currently under development. We’ll also make sure that in the production environment, the message of the day is set to a suitable message of the day for production. Create the file chef-playground/environments/production.json with the code provided in Example 15-2.

Example 15-2. chef-playground/environments/production.json

{
  "name": "production",
  "description": "For prods!",
  "cookbook_versions": {
    "apache": "= 0.1.0"
  },
  "json_class": "Chef::Environment",
  "chef_type": "environment",
  "override_attributes": {
    "motd": {
      "message": "A production-worthy message of the day"
    }
  }
}

Then run knife environment from file passing in the production.json file, as follows:

$ knife environment from file production.json
Updated Environment production

knife show environment displays more detailed information about an environment:

$ knife environment show production
chef_type:           environment
cookbook_versions:
  apache: = 0.1.0
default_attributes:
description:         For prods!
json_class:          Chef::Environment
name:                production
override_attributes:
  motd:
    message: A production-worthy message of the day

Things get a little more complicated with attribute precedence when environments come into the picture.Figure 15-2 shows that an environment has a priority less than a role, but greater than a cookbook recipe or attribute file.

Figure 15-2. Environment precedence

Because an environment can override and pin specific cookbook versions, it seems more reasonable that environments should have a higher priority than the default. So in this particular instance with environments, it does make sense to use override_attributes instead of default_attributes for any environment attributes.

Putting All the Pieces Together

Let’s go through a complete example using the apache cookbook from Chapter 7, making use of environments and roles as you would in a production Chef setup.

SIMULATE A PRODUCTION ENVIRONMENT

Create a directory called chef-zero. This will be structured similar to chef-repo in Chapter 9 and chef-playground, with cookbooks, environments, and roles as subdirectories. Once you create the directory, make it the current working directory as follows:

$ mkdir chef-zero
$ cd chef-zero

Create a chef-zero/environments subdirectory to contain our environment definitions in the JSON file format, and make it the current working directory:

$ mkdir environments
$ cd environments

Let’s say our apache cookbook is ready to go and we are making use of attributes, environments, and roles in our production environment. First, let’s simulate the production environment with Test Kitchen.

Create an environment definition in chef-zero/environments as shown in Example 15-3. This will represent a production environment, like we covered earlier in this chapter. There is also an attribute set for node['motd']['message'] as an attribute with override precedence.

Example 15-3. chefdk/chef-zero/environments/production.json

{
  "name": "production",
  "description": "For prods!",
  "cookbook_versions": {
    "apache": "= 0.1.0"
  },
  "json_class": "Chef::Environment",
  "chef_type": "environment",
  "override_attributes": {
    "motd": {
      "message": "A production-worthy message of the day"
    }
  }
}

Our production environment uses roles. Create a directory parallel to environments called roles, and make it the current working directory, as follows:

$ cd ..
$ mkdir roles
$ cd roles

In our production environment, we use a webserver role just like we covered in Chapter 14 to denote nodes that are web servers. Create the file chef-zero/roles/webserver.json as shown in Example 15-4. It contains the apachedirectory in its run list and the attribute node['apache']['port'] set at the default attribute precedence. We’ll be using this attribute to show how we can change the behavior of the cookbook depending on whether the node is in production.

Example 15-4. chefdk/chef-zero/roles/webserver.json

{
  "name": "webserver",
  "description": "Web Server",
  "json_class": "Chef::Role",
  "chef_type": "role",
  "default_attributes": {
    "apache": {
      "port": 80
    }
  },
  "run_list": [
    "recipe[apache]"
  ]
}

Create a directory called cookbooks, parallel to the others you have created so far in this chapter, and make it the current working directory:

$ cd ..
$ mkdir cookbooks
$ cd cookbooks

So far, your chef-zero directory should resemble the following structure:

chef-zero
    ├── cookbooks
    ├── environments
    │   └── production.json
    └── roles
        └── webserver.json

We’ll be recreating a version of the apache cookbook for this chapter, with a few additions. Create an apachecookbook in the cookbooks subdirectory by using chef generate cookbook or knife cookbook create, as per your Chef development tool setup.

Chef Development Kit:

$ chef generate cookbook apache
$ cd apache

Chef Client:

$ knife cookbook create apache --cookbook-path .
$ cd apache
$ kitchen init --create-gemfile
$ bundle install

Create a .kitchen.yml file as shown in Example 15-5. There’s a lot more going on in this version than we have seen in previous chapters.

For this example, we must use the chef_zero provisioner because we are making use of Chef Server features, so make sure that is being set in the provisioner: stanza of the .kitchen.yml. We need Test Kitchen to spin up a Chef Zero instance for us.

Also in the provisioner: stanza, we tell Test Kitchen where the roles and environments directories are relative to the location of the .kitchen.yml:

provisioner:
  name: chef_zero
  environments_path: ../../environments
  roles_path: ../../.roles

Don’t miss that we are setting the Test Kitchen suite name to be prod in the suites: stanza. We’re using a special suite name in this chapter because eventually there are going to be two suites, one for each environment, and we need a way of telling them apart. The suite for production will have the suite name prod.

We’re also introducing some new syntax in the suites: stanza. We set the environment for our sandbox node in its /etc/chef/client.rb, like so:

suites:
  - name: prod
    provisioner:
      client_rb:
        environment: production
  ...

Nodes can be a member of only one environment at a time. The environment is a setting in the /etc/chef/client.rbfile. If this is not set, a node uses the default environment named _default.

Outside of this simulated setup, you would use the chef-client::config recipe to change the value of the environment setting in /etc/chef/client.rb, using the following node attribute, similar to how we set ssl_verify_mode in Chapter 10:

node.default['chef_client']['config']['environment'] = 'production'

We also show how a private_network IP address can be set in the suites: stanza instead of the provisioner: stanza:

suites:
  - name: prod
  ...
  driver:
    network:
    - ["private_network", {ip: "192.168.33.15"}]

When a value is set in the provisioner: stanza in Test Kitchen, the values are inherited by all the items in the suites: stanza. In this case, we’re going to want our production and dev sandbox environments to have different IP addresses, so we move the private_network setting to be under suites.

Example 15-5. chefdk/chef-zero/cookbooks/apache/.kitchen.yml

---
driver:
  name: vagrant

provisioner:
  name: chef_zero
  environments_path: ../../environments
  roles_path: ../../.roles

platforms:
  - name: centos65
    driver:
      box: learningchef/centos65
      box_url: learningchef/centos65

suites:
  - name: prod
    provisioner:
      client_rb:
        environment: production
    driver:
      network:
      - ["private_network", {ip: "192.168.33.15"}]
    run_list:
      - recipe[apache::default]
    attributes:

Check the syntax of your .kitchen.yml with kitchen list. The output should resemble the following:

$ kitchen list
Instance       Driver   Provisioner  Last Action
prod-centos65  Vagrant  ChefZero     <Not Created>

Edit apache/metadata.rb, filling in the maintainer, maintainer_email, and license. We filled in ours like in Example 15-6.

Example 15-6. chefdk/chef-zero/cookbooks/apache/metadata.rb

name             'apache'
maintainer       'Mischa Taylor'
maintainer_email 'mischa@misheska.com'
license          'MIT'
description      'Installs/Configures apache'
long_description 'Installs/Configures apache'
version          '0.1.0'

Because we will be using attributes in this version of the apache cookbook, create a default.rb attributes file.

Chef Development Kit:

$ chef generate attribute default

Chef Client:

$ touch attributes/default.rb

Provide default settings for all the attributes we’re going to be using in our cookbook by creating attributes/default.rb as shown in Example 15-7. In order to test attribute precedence, we’re going to set the default values for node['apache']['port'] and node['motd']['message'] to something different than what is being set in the role and in the environment we are using. We also moved the root location for our index.html file to an attribute.

Example 15-7. chefdk/chef-zero/apache/attributes/default.rb

default['apache']['document_root'] = '/var/www/html'
default['apache']['port'] = 3333
default['motd']['message'] = 'Default message'

Create the recipe file recipes/default.rb as shown in Example 15-8. Most of this recipe code should look familiar.

We are adding a new template resource to create a custom.conf file on the sandbox node, along with an accompanying directory resource to create the required directory on the node. custom.conf is an optional file used to configure apache web server settings. In this file we’re going to set the default listening port and the document root.

We are introducing an alternative template resource syntax:

template '/etc/httpd/conf.d/custom.conf' do
  ...
  variables(
    :document_root => node['apache']['document_root'],
    :port => node['apache']['port']
  )
  ...
end

We covered the use of notifies in Chapter 9.

You can pass a hash of variables to be used when the template file is evaluated using the variables() attribute. This is a way to pass local instance variables in a recipe to a template, or to use shorter, more memorable variable names in the template file.

Example 15-8. chefdk/chef-zero/apache/recipes/default.rb

#
# Cookbook Name:: apache
# Recipe:: default
#
# Copyright (C) 2014
#
#
#

package 'httpd'

service 'httpd' do
  action [ :enable, :start ]
end

# Add a template for Apache virtual host configuration
template '/etc/httpd/conf.d/custom.conf' do
  source 'custom.erb'
  mode '0644'
  variables(
    :document_root => node['apache']['document_root'],
    :port => node['apache']['port']
  )
  notifies :restart, 'service[httpd]'
end

document_root = node['apache']['document_root']

# Add a directory resource to create the document_root
directory document_root do
  mode '0755'
  recursive true
end

template "#{document_root}/index.html" do
  source 'index.html.erb'
  mode '0644'
  variables(
    :message => node['motd']['message'],
    :port => node['apache']['port']
  )
end

Generate the template file templates/default/index.html.erb, using the appropriate command line for your Chef development setup.

Chef Development Kit:

$ chef generate template index.html

Chef Client - Linux/Mac OS X:

$ touch templates/default/index.html.erb

Chef Client - Windows:

$ touch templates\default\index.html.erb

Create the file templates/default/index.html.erb as shown in Example 15-9. We are using the short variable instance forms we defined in the variables() attribute of the template resource. Also, for some variety, we left one of them as the standard form: node["ipaddress"]. You can mix and match these forms as you like.

Example 15-9. chefdk/chef-zero/apache/templates/default/index.hmtl.erb

<html>
  <body>
    <h1><%= @message %></h1>
    <%= node["ipaddress"] %>:<%= @port %>
  </body>
</html>

Generate one more template file, templates/default/custom.erb, which will be used as an apache configuration file.

Chef Development Kit:

$ chef generate template custom

Chef Client - Linux/Mac OS X:

$ touch templates/default/custom.erb

Chef Client - Windows:

$ touch templates\default\custom.erb

Create templates/default/custom.erb as shown in Example 15-10. We’re using this optional apache configuration file to set the port the server is listening on via the Listen setting and the DocumentRoot.

We will explain the if syntax in the template in more detail in Chapter 16. In short for now, you can place conditional logic in templates when it is enclosed by <% %> (vs. <%= %> when you want to evaluate a string). Also, if the closing tag has a minus sign in it, such as -%>, the line is removed from the resultant template output when it is evaluated. Therefore, in Example 15-10, these three lines are processed when the template file is evaluated: <% if @port != 80 -%>, Listen <%= @port %>, and <% end -%>. When the evaluated output is written to the resultant template file, it becomes just one line, because there are -%> symbols on the first and the third lines: Listen <%= @port %>. Further, the single line with Listen <%= @port %> is only written to the resultant template file if the conditional logic evaluates to a port number besides 80.

We need this conditional logic in the template because the Listen line is required in the .conf file when any port besides 80 is used. If we left out the conditional, we’d get an error configuring the website if it evaluates to port 80.

Example 15-10. chefdk/chef-zero/apache/templates/default/custom.erb

<% if @port != 80 -%> 
  Listen <%= @port %> 
<% end -%> 

<VirtualHost *:<%= @port %>>
  ServerAdmin webmaster@localhost

  DocumentRoot <%= @document_root %>
  <Directory />
    Options FollowSymLinks
    AllowOverride None
  </Directory>
  <Directory <%= @document_root %>>
    Options Indexes FollowSymLinks MultiViews
    AllowOverride None
    Order allow,deny
    allow from all
  </Directory>
</VirtualHost>

This line is omitted from the resultant template file.

Only this line is written to the resultant template file.

This line is omitted from the resultant template file.

Run kitchen converge to deploy your cookbook to the sandbox node using the production environment:

$ kitchen converge prod-centos65

If all goes well, you should be able to view the production website on the sandbox node at http://192.168.33.15on the default web port 80—it should resemble Figure 15-3. The port 80 setting in the role overrides the default attribute set in the apache cookbook. Also, the message attribute set in the environment takes precedence.

Figure 15-3. Production web server

SIMULATE A DEVELOPMENT ENVIRONMENT

Let’s say we want to start a new development cycle for our apache cookbook, adding some new requested functionality. For the purposes of this chapter, we don’t care what the enhancements are, we just want our new cookbook development not to interfere with the stable 0.1.0 version we already have in production. We will perform our development on a node allocated to an environment called dev.

First, before doing anything else, increment the current cookbook version 0.1.0 to the next minor version number 0.2.0. We recommend that you follow semantic versioning guidelines when you version your cookbooks, incrementing the second digit when there are new changes that won’t break existing functionality. This is the intent with these hypothetical changes we might make to the apache cookbook.

Example 15-11. chefdk/chef-zero/cookbooks/apache/metadata.rb

name             'apache'
maintainer       'Mischa Taylor'
maintainer_email 'mischa@misheska.com'
license          'MIT'
description      'Installs/Configures apache'
long_description 'Installs/Configures apache'
version          '0.2.0'

If you try to deploy this new cookbook to the production node, you should get an error saying could not satisfy version constraints. Now we know that our production environment is enforcing the policy we set to pin the apache cookbook to version 0.1.0. When we tried to deploy version 0.2.0, we got an error:

$ kitchen converge prod-centos65
...
       Missing Cookbooks:
       ------------------
       Could not satisfy version constraints for: apache
...
Chef Client failed. 0 resources updated in 1.626076356 seconds
       [2014-08-22T17:59:26-07:00] ERROR: 412 "Precondition Failed "
       [2014-08-22T17:59:26-07:00] FATAL: Chef::Exceptions::ChildConvergeError:
       Chef run process exited unsuccessfully (exit code 1)
>>>>>> Converge failed on instance <prod-centos65>.
>>>>>> Please see .kitchen/logs/prod-centos65.log for more
details
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: SSH exited (1) for command: [sudo -E
chef-client -z --config /tmp/kitchen/client.rb --log_level info
--chef-zero-port 8889 --json-attributes /tmp/kitchen/dna.json]
>>>>>> ----------------------

Add a new environment definition to chef-zero/environments as shown in Example 15-12. We’ll pin the environment to use the latest cookbook version 0.2.0. Also, set the node['apache']['port'] and node['motd']['message'] to use developer-specific overrides.

Example 15-12. chefdk/chef-zero/environments/dev.json

{
  "name": "dev",
  "description": "For developers!",
  "cookbook_versions": {
    "apache": "= 0.2.0"
  },
  "json_class": "Chef::Environment",
  "chef_type": "environment",
  "override_attributes": {
    "apache": {
      "port": 8080
    },
    "motd": {
      "message": "Developers, developers, developers!"
    }
  }
}

Add a new dev suite to chef-zero/cookbooks/apache/.kitchen.yml as shown in Example 15-13. It’s in the same format as our prod instance—it just uses the dev environment and the IP address 192.168.33.16.

Example 15-13. chefdk/chef-zero/cookbooks/apache/.kitchen.yml

---
driver:
  name: vagrant

provisioner:
  name: chef_zero
  environments_path: ../../environments
  roles_path: ../../.roles

platforms:
  - name: centos65
    driver:
      box: learningchef/centos65
      box_url: learningchef/centos65

suites:
  - name: prod
    provisioner:
      client_rb:
        environment: production
    driver:
      network:
      - ["private_network", {ip: "192.168.33.15"}]
    run_list:
      - recipe[apache::default]
    attributes:

  - name: dev
    provisioner:
      client_rb:
        environment: dev
    driver:
      network:
      - ["private_network", {ip: "192.168.33.16"}]
    run_list:
      - recipe[apache::default]
    attributes:

Run kitchen list to check your .kitchen.yml syntax. Now you should see two instances, like so:

$ kitchen list
Instance       Driver   Provisioner  Last Action
prod-centos65  Vagrant  ChefZero     Converged
dev-centos65   Vagrant  ChefZero     <Not Created>

Run kitchen converge against the dev-centos65 instance, as follows:

$ kitchen converge dev-centos65

If all goes well, you should be able to view the development website on the sandbox node at http://192.168.33.16:8080—it should resemble Figure 15-4. The port 8080 setting in the environment overrides the default attribute set in the apache cookbook and in the role. Also, the message attribute set in the environment takes precedence.

Figure 15-4. Development web server

Run kitchen destroy with no parameters to destroy the virtual machines associated with both of the sandbox instances and to release all resources used:

$ kitchen converge dev-centos65
-----> Starting Kitchen (v1.2.2.dev)
-----> Destroying <prod-centos65>...
       ==> default: Forcing shutdown of VM...
       ==> default: Destroying VM and associated drives...
       Vagrant instance <prod-centos65> destroyed.
       Finished destroying <prod-centos65> (0m3.00s).
-----> Destroying <dev-centos65>...
       ==> default: Forcing shutdown of VM...
       ==> default: Destroying VM and associated drives...
       Vagrant instance <dev-centos65> destroyed.
       Finished destroying <dev-centos65> (0m3.00s).
-----> Kitchen is finished. (0m6.49s)

Summary

In this chapter we have covered environments and how they provide the ability to fix settings to match the different stages of your deployment workflow. We also walked you through a realistic example that made use of both environments and roles to change cookbook behavior. You can use environments to match the promotion model you use as your Chef code travels from development to production.

In the next, and final, chapter of this book, we’re going to show you how to test your automation code.