Tuesday, 3 January 2017

Appendix B. Hosted Enterprise Chef

Hosted Enterprise Chef is the cloud-based version of Chef Server. You can sign up for Hosted Enterprise Chef at no cost to you to manage up to five nodes, more than enough for getting started with Chef Server.
Go to https://manage.opscode.com/signup to sign up for a Hosted Enterprise Chef account. Complete the required fields.
Upon submitting the form, you will be redirected to a page with helpful links and documentation. Click on the Download Starter Kit button as shown in Figure B-1. This will download a ZIP file containing all your authentication keys, as well as a pre-populated Chef repo.
Extract the archive and move the chef-repo folder into your home directory. The chef-repo directory should resemble the following:
chef-repo/
├── .chef
│   ├── <username>.pem
│   ├── <organization>-validator.pem
│   └── knife.rb
├── .gitignore
├── README.md
├── Vagrantfile
├── cookbooks
└── roles
Hosted Enterprise Chef post-signup page
Figure B-1. Hosted Enterprise Chef post-signup page

NOTE

If you do not see a .chef directory, you might need to enable the display of system files in your file manager. Because the .chef directory starts with a dot, some operating systems will hide its presence.
The .chef directory contains three important files:
  • <username>.pem
  • <organization>-validator.pem
  • knife.rb
<username> is the username you used when you signed up for your Hosted Chef Server account.<organization> is the name of the company or organization you used when signing up for Hosted Chef Server. If you registered with the username “seth” and the organization “houseofbacon”, your .chef directory would contain:
  • seth.pem
  • houseofbacon.pem
  • knife.rb
The <username>.pem file is a unique identifier used to authenticate you against Chef Server. This should be treated like a password; do not share it with anyone, and do not alter the contents of the file.
The <organization>.pem file is a unique identifier used to authenticate your organization against Chef Server. This should be treated like a password, but it must also be shared among all your Chef developers. Anyone needing access to your Chef organization will also need a copy of this file. Do not alter the contents of this file, either.
Unlike the .pem files, the knife.rb file is meant to be edited, altered, and customized. The knife.rb file is recognized as Ruby and read by Chef when it issues commands:
current_dir = File.dirname(__FILE__)
log_level                :info
log_location             STDOUT
node_name                "<username>"
client_key               "#{current_dir}/<username>.pem"
validation_client_name   "<organization>-validator"
validation_key           "#{current_dir}/<organization>-validator.pem"
chef_server_url          "https://api.opscode.com/organizations/<organization>"
cache_type               'BasicFile'
cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )
cookbook_path            ["#{current_dir}/../cookbooks"]
As you can see, the knife.rb file sets some default configuration values, such as the log level, caching options, and cookbook paths. Additionally, the knife.rb configures the files client_keyvalidation_client_name, and validation_key. These values were automatically configured when you downloaded your starter kit. The chef_server_url configures the endpoint for Chef Server. Because we are using Hosted Enterprise Chef, this points to Opscode’s Chef API. If you were using Enterprise Chef or Open Source Chef Server, the URL would point to your Chef Server installation.
You can add other values to the knife.rb, such as EC2 credentials, proxy information, and encrypted data bag settings. Although certain pieces of the knife.rb will be common across your team, the contents of the file generally should be unique to you and your machine. However, unless you have access keys and passwords in your knife.rb, you do not need to treat it like a password.

Testing the Connection

You should run the following commands from inside the Chef repo. Open your terminal or command prompt, and make chef-repo the current working directory. If you placed your Chef repo in a different location, use that instead:
$ cd ~/chef-repo
Now you can use knife, the command-line tool for Chef Server, to test your connection and authentication against Chef Server. At the time of this writing, Chef does not provide a “connection test” command. However, asking Chef Server to list the clients will verify:
  • Your network can connect to Chef Server.
  • The authentication files are in the correct location.
  • The authentication files can be read by Chef.
  • The response from Chef Server is received by your workstation.
Issue the knife client list command on your terminal:
$ knife client list
Depending on the speed of your Internet connection, it might take a few seconds to get a response, but you should see the following:
$ knife client list
<organization>-validator
For example, if I named my Chef organization houseofbacon, the output would be:
$ knife client list
houseofbacon-validator
If you get an error, check the following:
  1. You can access https://api.opscode.com from a web browser.
  2. You are running commands from inside the chef-repo.
  3. The .chef directory contains two .pem files and a knife.rb.
  4. Your authentication files have the correct file permissions.
If you have confirmed the preceding steps and are still unable to connect to Chef Server, please consult the Chef online documentation.

Appendix A. Open Source Chef Server

This appendix is a variant of the instructions provided in Chapter 9 for Open Source Chef Server.

How to Install Open Source Chef Server Manually

First, download the Chef Server install package from http://www.getchef.com/chef/install. We’ll be installing Open Source Chef server on CentOS 6.5, so choose “Enterprise Linux > 6 > x86_64”, as shown in Figure A-1. Choose to Download version 11.1.4 to get a download link to the package.

NOTE

The download page might not match the images in this book exactly. However, the download and installation procedure should be the same.
The remaining steps necessary to install Chef Server are displayed below the download link:
  1. Install the chef-server package.
  2. Run sudo chef-server-ctl reconfigure.
Download Open Source Chef Server installation package
Figure A-1. Download Open Source Chef Server installation package

Install Open Source Chef Server

Assuming you have sufficient resources to install Chef Server locally along with a test node, let’s create a chef-server cookbook, which we’ll use to install Open Source Chef Server. To maintain consistency with Hosted Enterprise Chef, create the directory chef-repo/cookbooks and create the chef-server cookbook in that directory.
Create the chef-repo/cookbooks directory, and make it the current working directory.
Linux/Mac OS X:
$ mkdir -p chef-repo/cookbooks
$ cd chef-repo/cookbooks
Windows:
$ mkdir chef-repo\cookbooks
$ cd chef-repo\cookbooks
Now generate the chef-server cookbook with chef generate cookbook or knife cookbook create, depending on whether you are using the Chef Development Kit or the Chef Client, respectively. Also, enable the cookbook to use Test Kitchen. We’re going to go through the cookbook creation steps quickly in this chapter. If you need a refresher on what each of these commands mean and the expected output, refer back to Chapter 7.
Chef Development Kit:
$ chef generate cookbook chef-server
$ cd chef-server
Chef Client:
$ knife cookbook create chef-server --cookbook-path .
$ cd chef-server
$ kitchen init --create-gemfile
$ bundle install
Edit the .kitchen.yml file to use the CentOS 6.5 basebox we prepared specifically for this book. Also, assign a private network address like we did in Chapter 7. This time, we’re going to use the IP address 192.168.33.36. If this conflicts with an address already being used on your local network, change it to be a nonconflicting one. We also need more memory than the default 512 MB Test Kitchen allocates, so add a customize: block with a memory: statement to increase the memory to 1.5 GB (memory is specified in megabytes only).

NOTE

Make sure you use the chef_solo provisioner for this cookbook, as the in-memory Chef Server that the chef_zero provisioner spawns will cause a conflict with the hands-on exercises coming up in Chapter 10. As of this writing, if you want to automate the installation of a Chef Server with Chef cookbooks, the use of Chef Solo is recommended so that the deployment code doesn’t get confused by the presence of the in-memory Chef Server used in Chef Zero.
Example A-1. /chefdk/chef-repo/cookbooks/chef-server/.kitchen.yml
---
driver:
  name: vagrant

provisioner:
  name: chef_solo

platforms:
  - name: centos65
    driver:
      box: learningchef/centos65
      box_url: learningchef/centos65
      network:
      - ["private_network", {ip: "192.168.33.36"}]

suites:
  - name: default
    run_list:
      - recipe[chef-server::default]
    attributes:
Generate a default attributes file in attributes/default.rb.
Chef Development Kit:
$ chef generate attribute default
Chef Client:
$ touch attributes/default.rb
Add an attribute specifying the download URL for the Chef Server package that you obtained from http://www.getchef.com/chef/install. We recommend using the 11.1.4 version URL shown in Example A-2, as we wrote the examples for this chapter using this version.
Example A-2. /chefdk/chef-repo/cookbooks/chef-server/attributes/default.rb
default[chef-server][url] = \
https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64\
/chef-server-11.1.4-1.el6.x86_64.rpm
Let’s take an initial stab at coding a recipe to replicate the manual steps required to install Chef Server outlined in How to Install Open Source Chef Server Manually. Enter in the first version of the code as shown in Example A-3. Let’s go over some of the highlights of the code in the following paragraphs.
Rather than typing in long variable names such as node['chef-server']['url'], feel free to use temporary local variables in a recipe with shorter names, such as:
package_url = node['chef-server']['url']
Remember that you have the full power of Ruby classes and methods available to you in your Chef recipes, so don’t be afraid to use it. For example, you can use the ::File.basename() method to extract the package name from the URL. The package name is the last component of the URL after the forward slash (“/”): chef-server-11.1.4-1.el6.x86_64.rpm. Refer to the Ruby core API documentation for more information on the ::File class:
package_name = ::File.basename(package_url)
Unfortunately, the package resource does not work with URLs, so we’re introducing a new resource, the remote_file resource. The remote_file resource will download files from a remote location. Rather than hardcoding a path like “/tmp” for the package download, Chef provides a variable you should use instead: Chef::Config[:file_cache_path]. Let Chef choose the best place to store temporary files for you. Pass the local path where you want to store the file as a string parameter to remote_file or as a name attribute; in this case, we use the package_local_path variable. The download URL should be passed to remote_fileas the source attribute.
The package resource should be familiar to you by now—we used it in Chapter 7.
In order to execute the chef-server-ctl reconfigure, we need to introduce another resource, the execute resource. When you fail to find a resource that meets your needs, you can use the execute resource to run arbitrary shell commands. Pass the shell command you want to execute as a string parameter to the executeresource.
The full code is in Example A-3.
Example A-3. /chefdk/chef-repo/cookbooks/chef-server/recipes/default.rb
#
# Cookbook Name:: chef-server
# Recipe:: default
#
# Copyright (C) 2014
#
#
#

package_url = node['chef-server']['url']
package_name = ::File.basename(package_url)
package_local_path = "#{Chef::Config[:file_cache_path]}/#{package_name}"

# omnibus_package is remote (i.e., a URL) let's download it
remote_file package_local_path do
  source package_url
end

package package_local_path

# reconfigure the installation
execute 'chef-server-ctl reconfigure'
Run kitchen converge to install Chef Server, and use kitchen login to verify that the chef-serverpackage was installed:
$ kitchen converge default-centos65
$ kitchen login default-centos65
[vagrant@default-centos65 ~]$ rpm -q chef-server
chef-server-11.1.4-1.el6.x86_64
[vagrant@default-centos65 ~]$ exit
logout
Connection to 127.0.0.1 closed.

Introducing Idempotence

Although the recipe we created in Example A-3 is a good first attempt, it is not idempotent. When Chef code is idempotent, it can be run multiple times on the same system and the results will always be identical, without producing unintended side effects. All Chef default resources are guaranteed to be idempotent with the exception of the execute resource.
execute resources generally are not idempotent, because most command-line utilities can be run only once. They assume that a human being is interacting with the system and understands the state of the system. For example, assuming the file /learningchef/file1.txt exists, the following mv command will work the first time it is run, but it will fail the second time:
$ mv /learningchef/file1.txt /file1.txt
A great way to test to see if your recipe is idempotent is to run kitchen converge twice. When a recipe has no unintended side effects, there should be 0 resources updated on the second run.
Does our recipe pass the idempotency test? Sadly, no. Here’s a sampling of the output from an initial kitchen converge:
$ kitchen converge default-centos65
-----> Starting Kitchen (v1.2.2.dev)
-----> Creating <default-centos65>...
...
       Starting Chef Client, version 11.14.2
       [2014-08-17T23:06:15-07:00] INFO: *** Chef 11.14.2 ***
       [2014-08-17T23:06:15-07:00] INFO: Chef-client pid: 2386
       [2014-08-17T23:06:16-07:00] INFO: Setting the run_list to ["recipe[chef
       -server::default]"] from CLI options
       [2014-08-17T23:06:16-07:00] INFO: Run List is [recipe[chef-server
       ::default]]
       [2014-08-17T23:06:16-07:00] INFO: Run List expands to [chef-server
       ::default]
       [2014-08-17T23:06:16-07:00] INFO: Starting Chef Run for default-centos65
       [2014-08-17T23:06:16-07:00] INFO: Running start handlers
       [2014-08-17T23:06:16-07:00] INFO: Start handlers complete.
       Compiling Cookbooks...
       Converging 3 resources
...
       [2014-08-17T23:10:39-07:00] INFO: Chef Run complete in 262.973873452
       seconds

       Running handlers:
       [2014-08-17T23:10:39-07:00] INFO: Running report handlers
       Running handlers complete
       [2014-08-17T23:10:39-07:00] INFO: Report handlers complete
       Chef Client finished, 3/3 resources updated in 264.583736069 seconds
       Finished converging <default-centos65> (4m48.72s).
-----> Kitchen is finished. (5m27.52s)
Here’s the output from the second run. Chef thinks there’s still stuff it needs to do—2/3 resources updated in this second run, when we expected it to be 0/3:
$ kitchen converge
-----> Starting Kitchen (v1.2.2.dev)
-----> Converging <default-centos65>...
...
       Converging 3 resources
       Recipe: chef-server::default
  * remote_file[/tmp/kitchen/cache/chef-server-11.1.4-1.el6.x86_64.rpm] action
  create       [2014-08-17T23:13:55-07:00] INFO: Processing remote_file[/tmp
  /kitchen/cache/chef-server-11.1.4-1.el6.x86_64.rpm] action create (chef-server
  ::default line 15)
        (up to date)
         * package[/tmp/kitchen/cache/chef-server-11.1.4-1.el6.x86_64.rpm]
         action install
       [2014-08-17T23:13:58-07:00] INFO: Processing package[/tmp/kitchen/cache
       /chef-server-11.1.4-1.el6.x86_64.rpm] action install (chef-server
       ::default line 19)

           - install version 11.1.4-1.el6 of package /tmp/kitchen/cache/chef
           -server-11.1.4-1.el6.x86_64.rpm
         * execute[chef-server-ctl reconfigure] action run[2014-08-17T23:14:08
         -07:00] INFO: Processing execute[chef-server-ctl reconfigure] action
         run (chef-server::default line 22)
       [2014-08-17T23:14:20-07:00] INFO: execute[chef-server-ctl reconfigure]
       ran successfully

           - execute chef-server-ctl reconfigure
       [2014-08-17T23:14:20-07:00] INFO: Chef Run complete in 24.635708065
       seconds

       Running handlers:
       [2014-08-17T23:14:20-07:00] INFO: Running report handlers
       Running handlers complete
       [2014-08-17T23:14:20-07:00] INFO: Report handlers complete
       Chef Client finished, 2/3 resources updated in 26.532340506 seconds
       Finished converging <default-centos65> (0m28.53s).
-----> Kitchen is finished. (0m29.20s)
We’ve shown that the package resource is idempotent. As we mentioned earlier, most default Chef resources are. Notice that the remote_file resource is idempotent. It is reporting (up to date).
There are some issues with the package and execute resources, however, as on the second kitchen converge run Chef:
  1. Reinstalled the rpm package, unnecessarily
  2. Executed chef-server-ctl reconfigure a second time
Let’s fix these idempotency issues in our code now. Example A-4 has the final idempotent version of the code.
The first issue is a common one that Chef developers encounter with the package resource when they try to install from a downloaded rpm instead of using a package repository. Instead of using a package one-liner for a downloaded rpm, you need to tell the package resource to explicitly use the RPM provider via the providerattribute, and you need to tell it where the source RPM is located using the source attribute, like so:
package package_name do
  source package_local_path
  provider Chef::Provider::Package::Rpm
end

NOTE

You can use the rpm_package short name to specify the RPM provider to the package resource, if you prefer. The following code is equivalent to the preceding code:
rpm_package package_name do
  source package_local_path
end
Fixing the second issue with the execute resource is a little more involved. That’s why you should prefer built-in Chef resources over using the execute resource, because it’s up to you to make the execute resources idempotent.
One way to fix this issue is with a not_if guard to the execute resource. Guards are used to make a resource idempotent by allowing the resource to test for a desired state, and if the desired state is present, the resource should do nothing. In this case, we’ll test to see if the chef-server package is already installed, by adding a not_if clause to the execute resource as follows. not_if will test to see if the exit code of the command is 0, and if so the resource does nothing. (If you need to test the reverse of this logic, which is more typical on Windows, there is also an only_if guard—take a look at http://bit.ly/common_functionality for more information:
execute "chef-server-ctl reconfigure" do
  not_if "rpm -q chef-server"
end
While this is a reasonable way to address the issue, it’s a little clunky. You have to figure out a way to detect whether Chef Server is installed, and the method used in the previous example is not very reliable. A better approach is to trigger the execute when the package resource installs the package. You can trigger events in other resources with a notifies statement.
In order to use notifies, we’ll need to change the execute resource statement a bit. First, you’ll want to change the resource so it does nothing by default when execute is evaluated during the Chef run by adding an action :nothing attribute. Also, you’ll want to move the actual command line explicitly to the commandattribute, so that you can use a short name to trigger the execute block. By default, the name passed to the execute resource as a string parameter is used as the command attribute, which is great for a self-documenting one-liner, but not so great when you want to trigger the command by name. So let’s transform the executeresource like so:
# reconfigure the installation
execute 'reconfigure-chef-server' do
  command 'chef-server-ctl reconfigure'
  action :nothing
end
Now add the notifies attribute as follows. The notifies attribute takes three parameters: an action, the name of the resource to notify, and a timer indicating when the action should be performed. As shown in the following example, we want to perform the :run action on the execute[reconfigure-chef-server] resource, and we want the action performed :immediately. For more information on notifies parameters, refer to the Chef documentation:
package package_name do
  source package_local_path
  provider Chef::Provider::Package::Rpm
  notifies :run, 'execute[reconfigure-chef-server]', :immediately
end
Example A-4 shows what the final version of our idempotent code looks like.
Example A-4. chef-server/chefdk/chef-repo/cookbooks/chef-server/recipes/default.rb
# Cookbook Name:: chef-server
# Recipe:: default
#
# Copyright (C) 2014
#
#
#

package_url = node['chef-server']['url']
package_name = ::File.basename(package_url)
package_local_path = "#{Chef::Config[:file_cache_path]}/#{package_name}"

# omnibus_package is remote (i.e., a URL) let's download it
remote_file package_local_path do
  source package_url
end

package package_name do
  source package_local_path
  provider Chef::Provider::Package::Rpm
  notifies :run, 'execute[reconfigure-chef-server]'
end

# reconfigure the installation
execute 'reconfigure-chef-server' do
  command 'chef-server-ctl reconfigure'
  action :nothing
end
Try running kitchen converge against this revised recipe, and note that it reports 0/2 resources updated, which is the result we are looking for: 0 resources updated after running kitchen converge for the second time without any other changes to the cookbook:
$ kitchen converge
-----> Starting Kitchen (v1.2.2.dev)
-----> Converging <default-centos65>...
...
       Converging 3 resources
       Recipe: chef-server::default
         * remote_file[/tmp/kitchen/cache/chef-server-11.1.4-1.el6.x86_64.rpm]
         action create[2014-08-17T23:18:31-07:00] INFO: Processing remote_file[
         /tmp/kitchen/cache/chef-server-11.1.4-1.el6.x86_64.rpm] action create
         (chef-server::default line 15)
        (up to date)
         * package[chef-server-11.1.4-1.el6.x86_64.rpm] action install[2014-08
         -17T23:18:34-07:00] INFO: Processing package[chef-server-11.1.4-1.el6
         .x86_64.rpm] action install (chef-server::default line 19)
        (up to date)
         * execute[reconfigure-chef-server] action nothing[2014-08-17T23:18:34
         -07:00] INFO: Processing execute[reconfigure-chef-server] action
         nothing (chef-server::default line 26)
        (skipped due to action :nothing)
       [2014-08-17T23:18:34-07:00] INFO: Chef Run complete in 2.725841396
       seconds

       Running handlers:
       [2014-08-17T23:18:34-07:00] INFO: Running report handlers
       Running handlers complete
       [2014-08-17T23:18:34-07:00] INFO: Report handlers complete
       Chef Client finished, 0/2 resources updated in 4.347711133 seconds
       Finished converging <default-centos65> (0m5.81s).
-----> Kitchen is finished. (0m6.28s)
Always check your recipes to see if they are idempotent before deploying them to production. If we had deployed the first version of this recipe in production, given that the chef-client usually runs on a periodic timer performing Chef runs, all our nodes would have been reinstalling the Chef Server package and reconfiguring the server every 15 minutes!

Configure Open Source Chef Server

If your Open Source Chef Server installed properly, you should be able to access the web admin console using the private_network IP address you configured in your .kitchen.yml. In our case, we used the address 192.168.33.36. After you dismiss a warning about the use of a self-signed SSL certificate, log in with the default admin username and password, as shown in Figure A-2.
Log in as admin
Figure A-2. Log in as admin
You will be immediately prompted to reset the admin user’s password as shown in Figure A-3. Enter in a new password for the admin user and then:
  1. Click on the Save User button.
  2. Click on the Create tab when the admin user’s public key is shown to create a user account for yourself.
Reset admin password
Figure A-3. Reset admin password
When you click on the Create tab, you will be prompted to enter in a username and password for your user account. Enter in your username and password of choice. Make sure the Admin checkbox is selected, then click on the Create User button (see Figure A-4.
Finally, as shown in Figure A-5, you must download a copy of your user key, as this is used as your password for Chef Server and will be displayed only once. Copy the contents of the Private Key to the clipboard, and save it as <your_username>.pem. For example, our username is misheska, so we saved the file as misheska.pem.
Create a user
Figure A-4. Create a user
In the chef-repo directory you created in Install Open Source Chef Server, create a subdirectory called .chef. Our chef-repo is in our home directory; change the following command if your chef-repo is located elsewhere:
$ cd $HOME/chef-repo
$ mkdir .chef
After creating the chef-repo/.chef directory, copy the <username>.pem file you just created.
You also need to get a copy of the /etc/chef-server/chef-validator.pem key from Chef Server. Run the following scp command to download the key as root. The password is vagrant:
$ scp root@192.168.33.36:/etc/chef-server/chef-validator.pem .
root@192.168.33.36's password: vagrant
chef-validator.pem                            100% 1675     1.6KB/s   00:00
Copy user key
Figure A-5. Copy user key
Finally, create a file in chef-repo/.chef called knife.rbExample A-5 shows what our knife.rb file looks like; change the node_name and client_key fields in your version of the file to match your username for <username>.pem. Also, if you used a different IP address for your Chef Server, change the chef_server_urlfield accordingly.
Example A-5. chefdk/chef-repo/.chef/knife.rb
current_dir = File.dirname(__FILE__)
log_level                :info
log_location             STDOUT
node_name                "misheska"
client_key               "#{current_dir}/misheska.pem"
validation_client_name   "chef-validator"
validation_key           "#{current_dir}/chef-validator.pem"
chef_server_url          "https://default-centos65.vagrantup.com:443"
cache_type               "BasicFile"
cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )
cookbook_path            ["#{current_dir}/../cookbooks"]
Once you have finished creating the <user>.pemchef-validator.pem, and knife.rb files, your chef-repo/.chefdirectory should resemble the following:
chef-repo/.chef
├── chef-validator.pem
├── knife.rb
└── misheska.pem
The .chef directory now contains three important files:
  • <username>.pem
  • chef-validator.pem
  • knife.rb
<username> is the username you created in the Chef Server web admin tool. The <username>.pem file is a unique identifier used to authenticate you against Chef Server. This should be treated like a password; do not share it with anyone, and do not alter the contents of the file.
The chef-validator.pem file is a unique identifier used to authenticate your organization against Chef Server. This should be treated like a password, but it must also be shared among all your Chef developers. Anyone needing access to your Chef organization will also need a copy of this file. Do not alter the contents of this file, either.
Unlike the .pem files, the knife.rb file is meant to be edited, altered, and customized. The knife.rb file is recognized as Ruby and read by Chef when it issues commands:
current_dir = File.dirname(__FILE__)
log_level                :info
log_location             STDOUT
node_name                "<username>"
client_key               "#{current_dir}/<username>.pem"
validation_client_name   "chef-validator"
validation_key           "#{current_dir}/chef-validator.pem"
chef_server_url          "https://default-centos65.vagrantup.com:443"
cache_type               "BasicFile"
cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )
cookbook_path            ["#{current_dir}/../cookbooks"]
As you can see, the knife.rb file sets some default configuration values, such as the log level, caching options, and cookbook paths. Additionally, the knife.rb configures the file client_keyvalidation_client_name, and validation_key. The URL points to your Chef Server installation.
Note that the chef_server_url_ field uses a fake DNS hostname of default-centos65.vagrantup.combecause that’s the hostname vagrant set up. If you try to visit the URL https://default-centos65.vagrantup.com:443, you will discover that it is not valid.
Chef Server requires that hosts have valid fully qualified domain names set up in your local domain name service (DNS). In production, you would have your Chef Server hostname configured in your Domain Name System (DNS) server before installing. Let’s add a temporary host entry for default-centos65.vagrantup.com in your local host database in lieu of making a DNS change, as we are just doing a book exercise.
Run one of the following commands to add a host entry. Following are the commands we ran on our machine. If you used an IP address other than 192.168.33.36, make sure it matches when you run the command.
Linux/Mac OS X:
$ sudo sh -c "echo '192.168.33.36 default-centos65.vagrantup.com' >> /etc/hosts"
Windows Command Prompt:
> echo 192.168.33.36 default-centos65.vagrantup.com >> \
%WINDIR%\System32\Drivers\Etc\Hosts
Windows PowerShell:
PS> ac -Encoding UTF8 $env:windir\system32\drivers\etc\hosts \
"192.168.33.36 default-centos65.vagrantup.com"
Now if you try to visit https://default-centos65.vagrantup.com in your web browser, your local host should think this is a valid hostname.
You can add additional values to the knife.rb, such as EC2 credentials, proxy information, and encrypted data bag settings. Although certain pieces of the knife.rb will be common among your team members, the contents of the file generally should be unique to you and your machine. However, unless you have access keys and passwords in your knife.rb, you do not need to treat it like a password.

Testing the Connection

You should run the following commands from inside the Chef repo. Open your terminal or command prompt and make chef-repo the current working directory. If you placed your Chef repo in a different location, use that instead:
$ cd ~/chef-repo
Now you can use knife, the command-line tool for Chef Server, to test your connection and authentication against Chef Server. At the time of this writing, Chef does not provide a “connection test” command. However, asking Chef Server to list the clients will verify:
  • Your network can connect to Chef Server.
  • The authentication files are in the correct location.
  • The authentication files can be read by Chef.
  • The response from Chef Server is received by your workstation.
Issue the knife client list command on your terminal. You should see the following:
$ knife client list
chef-validator
chef-webui
If you get an error, check the following:
  1. You can access https://192.168.33.36:443 from a web browser.
  2. You are running commands from inside the chef-repo.
  3. The .chef directory contains two .pem files and a knife.rb.
  4. Your authentication files have the correct file permissions (they should be only user readable).
If you have confirmed the preceding steps and are still unable to connect to Chef Server, please consult the Chef online documentation.
Now that you have verified that your host can connect to Chef Server, let’s create another cookbook for a node instance and register it to be managed by Chef Server. If you created Chef Server locally in a sandbox environment in this chapter, leave it running—we’ll be using it in the next chapter.

Bootstrapping a Node

In Chef, the term “bootstrapping” refers to the process by which a remote system is prepared to be managed by Chef. This process includes installing Chef Client and registering the target node with Chef Server.

Create a Node

Let’s use Test Kitchen to define a project that spins up a node in a sandbox environment, similar to what we did back in Chapter 5 before we learned how to create cookbooks.
Create a node directory alongside the chef-server cookbook you created in Chapter 9. This technically isn’t a cookbook—it’s just a Test Kitchen project—but putting it beside the chef-server cookbook directory makes it convenient to go back and forth between the two.
Create the directory ~/chef-repo/cookbooks/node, and make it the current working directory:
$ cd ~/chef-repo/cookbooks
$ mkdir node
$ cd node
Assuming that you set up knife to communicate with your Chef Server following either the instructions in Chapter 9 or Appendix B, the knife client list command should work even in this subdirectory. Verify this now:
$ knife client list
chef-validator
chef-webui
This node directory will just be a test kitchen project, not a cookbook, so run the following commands to create a .kitchen.yml file for Test Kitchen:
$ kitchen init --create-gemfile
$ bundle install
Edit the .kitchen.yml file to use the CentOS 6.5 basebox we prepared specifically for this book. Also assign a private network address like we did in Chapter 7. This time, we’re going to use the IP address 192.168.33.37. Make sure this address does not conflict with the IP address of your Chef Server. It should be 192.168.33.36.
Note that we also changed the suite name to be node, as this sandbox environment will be running our node, and have the other sandbox environment running our Chef Server. Having different names will disambiguate the two environments.
Example A-6. chefdk/chef-repo/cookbooks/node/.kitchen.yml
---
driver:
  name: vagrant

provisioner:
  name: shell

platforms:
  - name: centos65
    driver:
      box: learningchef/centos65
      box_url: learningchef/centos65
      network:
      - ["private_network", {ip: "192.168.33.37"}]

suites:
  - name: node
    run_list:
    attributes:
Spin up the node environment with kitchen create:
$ kitchen create node-centos65

Bootstrap the Node with Knife

Figure A-6 presents an overview of the current setup we’ve configured so far. We’ve configured a Chef Server (or used Hosted Enterprise Chef), and we configured a knife.rb with the appropriate keys, so that we can communicate with the Chef server from our host, the administrator’s workstation. We’ve established that this communication channel works by verifying that knife client list produces the expected output.
Now let’s set up our node like we would in production by “bootstrapping” the node with knife bootstrap. (We won’t be able to use Test Kitchen in production!) When we run knife bootstrap on the our host, it will install Chef Client on the node and register it to be managed by Chef Server.
Nodes must have valid, fully qualified domain names set up in your local domain name service (DNS) as well. Let’s add an entry to our local host database for the node just like we did for Chef Server.
Overview of our setup so far, before nodes
Figure A-6. Overview of our setup so far, before nodes
Run one of the following commands to add a node entry. Following are the commands we ran on our machine. If you used an IP address other than 192.168.33.35, make sure it matches when you run the command.
Linux/Mac OS X:
$ sudo sh -c "echo '192.168.33.37 node-centos65.vagrantup.com' >> /etc/hosts"
Windows Command Prompt:
> echo 192.168.33.37 node-centos65.vagrantup.com >> \
%WINDIR%\System32\Drivers\Etc\Hosts
Windows PowerShell:
PS> ac -Encoding UTF8 $env:windir\system32\drivers\etc\hosts \
"192.168.33.37 node-centos65.vagrantup.com"
You also need to kitchen login to the node and configure the local host database on the node to provide it the name of the server where you will run chef-client. Once this is complete, exit back out to your host prompt. As mentioned before, in production, you’d just make sure the DNS was configured with these hostnames before installing Chef Server and any nodes:
$ kitchen login node-centos65
Last login: Fri Jul  4 14:48:27 2014 from 10.0.2.2
Welcome to your Packer-built virtual machine.
[vagrant@node-centos65 ~]$ sudo sh -c "echo \
'192.168.33.36 default-centos65.vagrantup.com' >> /etc/hosts"
[vagrant@node-centos65 ~]$ exit
logout
Connection to 127.0.0.1 closed.
Run the following command to bootstrap your node:
$ knife bootstrap --sudo --ssh-user vagrant --ssh-password \
vagrant --no-host-key-verify node-centos65.vagrantup.com
Connecting to node-centos65.vagrantup.com
node-centos65.vagrantup.com Installing Chef Client...
...
node-centos65.vagrantup.com Thank you for installing Chef!
node-centos65.vagrantup.com Starting first Chef Client run...
...
node-centos65.vagrantup.com Starting Chef Client, version 11.14.2
node-centos65.vagrantup.com Creating a new client identity for
node-centos65.vagrantup.com using the validator key.
node-centos65.vagrantup.com resolving cookbooks for run list: []
node-centos65.vagrantup.com Synchronizing Cookbooks:
node-centos65.vagrantup.com Compiling Cookbooks...
node-centos65.vagrantup.com [2014-08-18T00:05:44-07:00]
WARN: Node node-centos65.vagrantup.com has an empty run list.
node-centos65.vagrantup.com Converging 0 resources
node-centos65.vagrantup.com
node-centos65.vagrantup.com Running handlers:
node-centos65.vagrantup.com Running handlers complete
node-centos65.vagrantup.com Chef Client finished, 0/0 resources updated in
2.595912205 seconds
You can tell from the output that it successfully installed Chef Client, and even performed a courtesy Chef Client run, but there were no cookbooks in the run list.
To verify that the node is now registered on the Chef server, log into the web interface and click on the Nodes tab. Now you should see that you have a node registered with your Chef Server, as shown in Figure A-7.
Node is registered with Chef Server
Figure A-7. Node is registered with Chef Server
If you click on the link to the node, you should see that Chef Server displays information about the node as shown in Figure A-8. The values you see under attributes should look familiar—these are the attributes generated automatically by ohai. They are stored on Chef Server for each node, and the data is searchable by all clients.
We are now done with both our Chef Server and node. Go ahead and kitchen destroy both of them:
$ cd ~/chef-repo/cookbooks/chef-server
$ kitchen destroy
$ cd ~/chef-repo/cookbooks/node
$ kitchen destroy

NOTE

Don’t forget to remove the entries for default-centos65.vagrantup.com and node-centos65.vagrantup.com from /etc/hosts on Linux/Mac OS X or %WINDIR%\system32\drivers\etc\hostson Windows.
Node is registered with Chef Server
Figure A-8. Node is registered with Chef Server