This appendix is a variant of the instructions provided in Chapter 9 for Open Source Chef Server.
How to Install Open Source Chef Server Manually
First, download the Chef Server install package from http://www.getchef.com/chef/install. We’ll be installing Open Source Chef server on CentOS 6.5, so choose “Enterprise Linux > 6 > x86_64”, as shown in Figure A-1. Choose to Download version 11.1.4 to get a download link to the package.
NOTE
The download page might not match the images in this book exactly. However, the download and installation procedure should be the same.
The remaining steps necessary to install Chef Server are displayed below the download link:
- Install the chef-server package.
- Run
sudo chef-server-ctl reconfigure.
Install Open Source Chef Server
Assuming you have sufficient resources to install Chef Server locally along with a test node, let’s create a
chef-server cookbook, which we’ll use to install Open Source Chef Server. To maintain consistency with Hosted Enterprise Chef, create the directory chef-repo/cookbooks and create the chef-server cookbook in that directory.
Create the chef-repo/cookbooks directory, and make it the current working directory.
Linux/Mac OS X:
Windows:
Now generate the
chef-server cookbook with chef generate cookbook or knife cookbook create, depending on whether you are using the Chef Development Kit or the Chef Client, respectively. Also, enable the cookbook to use Test Kitchen. We’re going to go through the cookbook creation steps quickly in this chapter. If you need a refresher on what each of these commands mean and the expected output, refer back to Chapter 7.
Chef Development Kit:
Chef Client:
Edit the .kitchen.yml file to use the CentOS 6.5 basebox we prepared specifically for this book. Also, assign a private network address like we did in Chapter 7. This time, we’re going to use the IP address 192.168.33.36. If this conflicts with an address already being used on your local network, change it to be a nonconflicting one. We also need more memory than the default 512 MB Test Kitchen allocates, so add a
customize: block with a memory: statement to increase the memory to 1.5 GB (memory is specified in megabytes only).NOTE
Make sure you use the
chef_solo provisioner for this cookbook, as the in-memory Chef Server that the chef_zero provisioner spawns will cause a conflict with the hands-on exercises coming up in Chapter 10. As of this writing, if you want to automate the installation of a Chef Server with Chef cookbooks, the use of Chef Solo is recommended so that the deployment code doesn’t get confused by the presence of the in-memory Chef Server used in Chef Zero.
Generate a default attributes file in attributes/default.rb.
Chef Development Kit:
Chef Client:
Add an attribute specifying the download URL for the Chef Server package that you obtained from http://www.getchef.com/chef/install. We recommend using the 11.1.4 version URL shown in Example A-2, as we wrote the examples for this chapter using this version.
Let’s take an initial stab at coding a recipe to replicate the manual steps required to install Chef Server outlined in How to Install Open Source Chef Server Manually. Enter in the first version of the code as shown in Example A-3. Let’s go over some of the highlights of the code in the following paragraphs.
Rather than typing in long variable names such as
node['chef-server']['url'], feel free to use temporary local variables in a recipe with shorter names, such as:
Remember that you have the full power of Ruby classes and methods available to you in your Chef recipes, so don’t be afraid to use it. For example, you can use the
::File.basename() method to extract the package name from the URL. The package name is the last component of the URL after the forward slash (“/”): chef-server-11.1.4-1.el6.x86_64.rpm. Refer to the Ruby core API documentation for more information on the ::File class:
Unfortunately, the
package resource does not work with URLs, so we’re introducing a new resource, the remote_file resource. The remote_file resource will download files from a remote location. Rather than hardcoding a path like “/tmp” for the package download, Chef provides a variable you should use instead: Chef::Config[:file_cache_path]. Let Chef choose the best place to store temporary files for you. Pass the local path where you want to store the file as a string parameter to remote_file or as a name attribute; in this case, we use the package_local_path variable. The download URL should be passed to remote_fileas the source attribute.
The
package resource should be familiar to you by now—we used it in Chapter 7.
In order to execute the
chef-server-ctl reconfigure, we need to introduce another resource, the execute resource. When you fail to find a resource that meets your needs, you can use the execute resource to run arbitrary shell commands. Pass the shell command you want to execute as a string parameter to the executeresource.
The full code is in Example A-3.
Introducing Idempotence
Although the recipe we created in Example A-3 is a good first attempt, it is not idempotent. When Chef code is idempotent, it can be run multiple times on the same system and the results will always be identical, without producing unintended side effects. All Chef default resources are guaranteed to be idempotent with the exception of the
execute resource.execute resources generally are not idempotent, because most command-line utilities can be run only once. They assume that a human being is interacting with the system and understands the state of the system. For example, assuming the file /learningchef/file1.txt exists, the following mv command will work the first time it is run, but it will fail the second time:
A great way to test to see if your recipe is idempotent is to run
kitchen converge twice. When a recipe has no unintended side effects, there should be 0 resources updated on the second run.
Does our recipe pass the idempotency test? Sadly, no. Here’s a sampling of the output from an initial
kitchen converge:
Here’s the output from the second run. Chef thinks there’s still stuff it needs to do—2/3 resources updated in this second run, when we expected it to be 0/3:
We’ve shown that the
package resource is idempotent. As we mentioned earlier, most default Chef resources are. Notice that the remote_file resource is idempotent. It is reporting (up to date).
There are some issues with the
package and execute resources, however, as on the second kitchen converge run Chef:- Reinstalled the rpm package, unnecessarily
- Executed
chef-server-ctl reconfigurea second time
Let’s fix these idempotency issues in our code now. Example A-4 has the final idempotent version of the code.
The first issue is a common one that Chef developers encounter with the package resource when they try to install from a downloaded
rpm instead of using a package repository. Instead of using a package one-liner for a downloaded rpm, you need to tell the package resource to explicitly use the RPM provider via the providerattribute, and you need to tell it where the source RPM is located using the source attribute, like so:NOTE
You can use the
rpm_package short name to specify the RPM provider to the package resource, if you prefer. The following code is equivalent to the preceding code:
Fixing the second issue with the
execute resource is a little more involved. That’s why you should prefer built-in Chef resources over using the execute resource, because it’s up to you to make the execute resources idempotent.
One way to fix this issue is with a
not_if guard to the execute resource. Guards are used to make a resource idempotent by allowing the resource to test for a desired state, and if the desired state is present, the resource should do nothing. In this case, we’ll test to see if the chef-server package is already installed, by adding a not_if clause to the execute resource as follows. not_if will test to see if the exit code of the command is 0, and if so the resource does nothing. (If you need to test the reverse of this logic, which is more typical on Windows, there is also an only_if guard—take a look at http://bit.ly/common_functionality for more information:
While this is a reasonable way to address the issue, it’s a little clunky. You have to figure out a way to detect whether Chef Server is installed, and the method used in the previous example is not very reliable. A better approach is to trigger the execute when the
package resource installs the package. You can trigger events in other resources with a notifies statement.
In order to use
notifies, we’ll need to change the execute resource statement a bit. First, you’ll want to change the resource so it does nothing by default when execute is evaluated during the Chef run by adding an action :nothing attribute. Also, you’ll want to move the actual command line explicitly to the commandattribute, so that you can use a short name to trigger the execute block. By default, the name passed to the execute resource as a string parameter is used as the command attribute, which is great for a self-documenting one-liner, but not so great when you want to trigger the command by name. So let’s transform the executeresource like so:
Now add the
notifies attribute as follows. The notifies attribute takes three parameters: an action, the name of the resource to notify, and a timer indicating when the action should be performed. As shown in the following example, we want to perform the :run action on the execute[reconfigure-chef-server] resource, and we want the action performed :immediately. For more information on notifies parameters, refer to the Chef documentation:
Example A-4 shows what the final version of our idempotent code looks like.
Try running
kitchen converge against this revised recipe, and note that it reports 0/2 resources updated, which is the result we are looking for: 0 resources updated after running kitchen converge for the second time without any other changes to the cookbook:
Always check your recipes to see if they are idempotent before deploying them to production. If we had deployed the first version of this recipe in production, given that the
chef-client usually runs on a periodic timer performing Chef runs, all our nodes would have been reinstalling the Chef Server package and reconfiguring the server every 15 minutes!Configure Open Source Chef Server
If your Open Source Chef Server installed properly, you should be able to access the web admin console using the
private_network IP address you configured in your .kitchen.yml. In our case, we used the address 192.168.33.36. After you dismiss a warning about the use of a self-signed SSL certificate, log in with the default admin username and password, as shown in Figure A-2.
You will be immediately prompted to reset the
admin user’s password as shown in Figure A-3. Enter in a new password for the admin user and then:- Click on the Save User button.
- Click on the Create tab when the admin user’s public key is shown to create a user account for yourself.
When you click on the Create tab, you will be prompted to enter in a username and password for your user account. Enter in your username and password of choice. Make sure the Admin checkbox is selected, then click on the Create User button (see Figure A-4.
Finally, as shown in Figure A-5, you must download a copy of your user key, as this is used as your password for Chef Server and will be displayed only once. Copy the contents of the Private Key to the clipboard, and save it as <your_username>.pem. For example, our username is misheska, so we saved the file as misheska.pem.
In the chef-repo directory you created in Install Open Source Chef Server, create a subdirectory called .chef. Our chef-repo is in our home directory; change the following command if your chef-repo is located elsewhere:
After creating the chef-repo/.chef directory, copy the <username>.pem file you just created.
You also need to get a copy of the /etc/chef-server/chef-validator.pem key from Chef Server. Run the following
scp command to download the key as root. The password is vagrant:
Finally, create a file in chef-repo/.chef called knife.rb. Example A-5 shows what our knife.rb file looks like; change the
node_name and client_key fields in your version of the file to match your username for <username>.pem. Also, if you used a different IP address for your Chef Server, change the chef_server_urlfield accordingly.
Once you have finished creating the <user>.pem, chef-validator.pem, and knife.rb files, your chef-repo/.chefdirectory should resemble the following:
The .chef directory now contains three important files:
- <username>.pem
- chef-validator.pem
- knife.rb
<username> is the username you created in the Chef Server web admin tool. The <username>.pem file is a unique identifier used to authenticate you against Chef Server. This should be treated like a password; do not share it with anyone, and do not alter the contents of the file.
The chef-validator.pem file is a unique identifier used to authenticate your organization against Chef Server. This should be treated like a password, but it must also be shared among all your Chef developers. Anyone needing access to your Chef organization will also need a copy of this file. Do not alter the contents of this file, either.
Unlike the .pem files, the knife.rb file is meant to be edited, altered, and customized. The knife.rb file is recognized as Ruby and read by Chef when it issues commands:
As you can see, the knife.rb file sets some default configuration values, such as the log level, caching options, and cookbook paths. Additionally, the knife.rb configures the file
client_key, validation_client_name, and validation_key. The URL points to your Chef Server installation.
Note that the
chef_server_url_ field uses a fake DNS hostname of default-centos65.vagrantup.combecause that’s the hostname vagrant set up. If you try to visit the URL https://default-centos65.vagrantup.com:443, you will discover that it is not valid.
Chef Server requires that hosts have valid fully qualified domain names set up in your local domain name service (DNS). In production, you would have your Chef Server hostname configured in your Domain Name System (DNS) server before installing. Let’s add a temporary host entry for
default-centos65.vagrantup.com in your local host database in lieu of making a DNS change, as we are just doing a book exercise.
Run one of the following commands to add a host entry. Following are the commands we ran on our machine. If you used an IP address other than
192.168.33.36, make sure it matches when you run the command.
Linux/Mac OS X:
Windows Command Prompt:
Windows PowerShell:
Now if you try to visit https://default-centos65.vagrantup.com in your web browser, your local host should think this is a valid hostname.
You can add additional values to the knife.rb, such as EC2 credentials, proxy information, and encrypted data bag settings. Although certain pieces of the knife.rb will be common among your team members, the contents of the file generally should be unique to you and your machine. However, unless you have access keys and passwords in your knife.rb, you do not need to treat it like a password.
Testing the Connection
You should run the following commands from inside the Chef repo. Open your terminal or command prompt and make chef-repo the current working directory. If you placed your Chef repo in a different location, use that instead:
Now you can use
knife, the command-line tool for Chef Server, to test your connection and authentication against Chef Server. At the time of this writing, Chef does not provide a “connection test” command. However, asking Chef Server to list the clients will verify:- Your network can connect to Chef Server.
- The authentication files are in the correct location.
- The authentication files can be read by Chef.
- The response from Chef Server is received by your workstation.
Issue the
knife client list command on your terminal. You should see the following:
If you get an error, check the following:
- You can access
https://192.168.33.36:443from a web browser. - You are running commands from inside the chef-repo.
- The .chef directory contains two .pem files and a knife.rb.
- Your authentication files have the correct file permissions (they should be only user readable).
If you have confirmed the preceding steps and are still unable to connect to Chef Server, please consult the Chef online documentation.
Now that you have verified that your host can connect to Chef Server, let’s create another cookbook for a node instance and register it to be managed by Chef Server. If you created Chef Server locally in a sandbox environment in this chapter, leave it running—we’ll be using it in the next chapter.
Bootstrapping a Node
Create a Node
Let’s use Test Kitchen to define a project that spins up a node in a sandbox environment, similar to what we did back in Chapter 5 before we learned how to create cookbooks.
Create a
node directory alongside the chef-server cookbook you created in Chapter 9. This technically isn’t a cookbook—it’s just a Test Kitchen project—but putting it beside the chef-server cookbook directory makes it convenient to go back and forth between the two.
Create the directory ~/chef-repo/cookbooks/node, and make it the current working directory:
Assuming that you set up
knife to communicate with your Chef Server following either the instructions in Chapter 9 or Appendix B, the knife client list command should work even in this subdirectory. Verify this now:
This node directory will just be a test kitchen project, not a cookbook, so run the following commands to create a .kitchen.yml file for Test Kitchen:
Edit the .kitchen.yml file to use the CentOS 6.5 basebox we prepared specifically for this book. Also assign a private network address like we did in Chapter 7. This time, we’re going to use the IP address 192.168.33.37. Make sure this address does not conflict with the IP address of your Chef Server. It should be 192.168.33.36.
Note that we also changed the suite name to be
node, as this sandbox environment will be running our node, and have the other sandbox environment running our Chef Server. Having different names will disambiguate the two environments.
Spin up the node environment with
kitchen create:Bootstrap the Node with Knife
Figure A-6 presents an overview of the current setup we’ve configured so far. We’ve configured a Chef Server (or used Hosted Enterprise Chef), and we configured a knife.rb with the appropriate keys, so that we can communicate with the Chef server from our host, the administrator’s workstation. We’ve established that this communication channel works by verifying that
knife client list produces the expected output.
Now let’s set up our node like we would in production by “bootstrapping” the node with
knife bootstrap. (We won’t be able to use Test Kitchen in production!) When we run knife bootstrap on the our host, it will install Chef Client on the node and register it to be managed by Chef Server.
Nodes must have valid, fully qualified domain names set up in your local domain name service (DNS) as well. Let’s add an entry to our local host database for the node just like we did for Chef Server.
Run one of the following commands to add a node entry. Following are the commands we ran on our machine. If you used an IP address other than
192.168.33.35, make sure it matches when you run the command.
Linux/Mac OS X:
Windows Command Prompt:
Windows PowerShell:
You also need to
kitchen login to the node and configure the local host database on the node to provide it the name of the server where you will run chef-client. Once this is complete, exit back out to your host prompt. As mentioned before, in production, you’d just make sure the DNS was configured with these hostnames before installing Chef Server and any nodes:
Run the following command to bootstrap your node:
You can tell from the output that it successfully installed Chef Client, and even performed a courtesy Chef Client run, but there were no cookbooks in the run list.
To verify that the node is now registered on the Chef server, log into the web interface and click on the Nodes tab. Now you should see that you have a node registered with your Chef Server, as shown in Figure A-7.
If you click on the link to the node, you should see that Chef Server displays information about the node as shown in Figure A-8. The values you see under attributes should look familiar—these are the attributes generated automatically by
ohai. They are stored on Chef Server for each node, and the data is searchable by all clients.
We are now done with both our Chef Server and node. Go ahead and
kitchen destroy both of them:
ReplyDeleteNice Blog!! It is very helpful and very informative and I really learned a lot from it.
DevOps Online Training institute
DevOps Online Training in Hyderabad