A Complete Beginner‘s Guide to Chef and Infrastructure as Code

Modern software systems can be incredibly complex, often spanning hundreds or thousands of servers. Manually configuring that infrastructure is time-consuming, error-prone, and simply not feasible at scale. That‘s where infrastructure as code (IaC) tools like Chef come in.

In this beginner‘s guide, we‘ll take a deep dive into Chef and show you how it can transform the way you deploy and manage infrastructure. Whether you‘re a developer, sysadmin, or DevOps engineer, understanding infrastructure as code is increasingly becoming an essential skill. Let‘s get started!

What is Infrastructure as Code?

Infrastructure as code (IaC) is the process of managing and provisioning computing infrastructure through machine-readable definition files or code, rather than manual, interactive configuration. IaC takes proven coding techniques and extends them to infrastructure, enabling you to treat your servers, databases, networks, and other infrastructure like software.

With infrastructure defined as code, you can:

• Provision infrastructure quickly and consistently by running the same code every time
• Automate deployments and reduce manual, repetitive tasks
• Enforce consistency and avoid configuration drift across environments
• Version control your infrastructure and track changes over time
• Adopt software development best practices like code review, continuous integration, and testing

Ultimately, IaC helps teams rapidly deliver applications and services at scale, while improving reliability and lowering operational overhead. Some popular IaC tools include Chef, Puppet, Ansible, and Terraform.

An Introduction to Chef

Chef is a powerful infrastructure automation platform that extends the principles of IaC to the entire stack, from server provisioning to application deployment to compliance audits. With Chef, you write code to describe the desired state of your infrastructure, then execute that code to configure systems accordingly.

Chef uses a Ruby-based DSL (domain-specific language), making it accessible to developers and sysadmins alike. Don‘t worry if you‘re not a Ruby expert – Chef only requires basic familiarity with the language.

Here are some key benefits and use cases of Chef:

• Consistency – Chef ensures every system is configured exactly as declared in code. No more configuration drift or snowflake servers.
• Scalability – Automate configuration on 10 servers or 10,000 servers with minimal additional effort. Chef makes it easy to elastically scale infrastructure up and down.
• Speed – Provision and configure systems much faster than doing it manually by hand. Accelerate deployments from weeks to minutes.
• Version control – Check infrastructure code into version control systems like Git to track changes, collaborate with team members, and maintain an audit trail.
• Continuous delivery – Integrate Chef into CI/CD pipelines to test infrastructure changes and deploy new versions alongside your application code.

Chef Architecture and Components

Before diving into Chef code, it‘s important to understand the key components that make up a typical Chef implementation:

• Chef Workstation – Your local machine where you author Chef code, test it, and interact with the Chef server. Chef code is packaged into artifacts called cookbooks.

• Chef Server – The central hub that stores your cookbooks and other configuration data. The Chef server also maintains an index of all registered nodes and their state. Nodes communicate with the Chef server to pull down the latest cookbooks.

• Chef Client (Nodes) – The machines being configured by Chef. The Chef client runs on each node and executes cookbooks to bring the node into the desired state. Clients poll the Chef server periodically to retrieve updated cookbooks.

In addition to these core components, Chef also includes tools like:

• Knife – A command-line tool for interacting with the Chef server and managing nodes, cookbooks, roles, environments, and more.
• Ohai – A tool that profiles your nodes and collects detailed system information like hardware specs, network configuration, operating system, and more. This data is accessible inside Chef recipes.
• Test Kitchen – A testing harness for Chef that lets you write automated tests and run cookbooks across multiple platforms.

With this high-level architecture in mind, let‘s walk through the process of getting started with Chef.

Setting Up Your Chef Environment

The first step is installing the Chef Development Kit (ChefDK) on your workstation. ChefDK is an omnibus package that includes all the tools you need to start coding with Chef.

To install ChefDK:

1. Visit the ChefDK downloads page
2. Select the appropriate package for your operating system and architecture
3. Run the installer

Once installed, open a terminal and run chef --version to verify ChefDK is set up correctly. You should see output like:

Chef Development Kit Version: 4.13.3
chef-client version: 16.6.14
delivery version: master (6862f27aba89109a9630f0b6c6798efec56b4efe)
berks version: 7.0.8
kitchen version: 2.7.0
inspec version: 4.24.8

Next, let‘s generate a starter cookbook using the chef command:

chef generate cookbook my_webserver

This creates a new directory named my_webserver with the following structure:

my_webserver/
├── Berksfile
├── CHANGELOG.md
├── chefignore
├── LICENSE
├── metadata.rb
├── Policyfile.rb
├── README.md
├── recipes
│   └── default.rb
├── spec
│   ├── spec_helper.rb
│   └── unit
│       └── recipes
│           └── default_spec.rb
└── test
    └── integration
        └── default
            └── default_test.rb

Some key files to note:

• recipes/default.rb – This is your default recipe where you‘ll write Chef code
• Policyfile.rb and Berksfile – Define cookbook dependencies
• metadata.rb – Specify cookbook name, version, maintainer, and other metadata
• spec/ and test/ – Directories for writing unit and integration tests (we won‘t cover testing in this guide)

Time to start writing some Chef code!

Writing Chef Resources and Recipes

The fundamental building blocks of Chef are resources and recipes. A resource represents some piece of infrastructure and its desired state – like a package that should be installed, a service that should be running, or a file that should exist. A recipe is a collection of resources that tells Chef how to configure a system.

Chef ships with dozens of built-in resources for managing packages, services, files, users, and much more. You declare resources in recipes using a Ruby-based DSL. Here‘s a simple recipe that installs the Apache web server on an Ubuntu system:

package ‘apache2‘ do
  action :install
end

service ‘apache2‘ do
  action [:enable, :start]
end

This recipe contains two resources:

1. A package resource to install the apache2 package
2. A service resource to enable and start the apache2 service

Save this code in recipes/default.rb. To apply this recipe locally, run:

chef-client --local-mode --runlist ‘recipe[my_webserver]‘

Chef will install Apache and start the service. It‘s that easy!

Let‘s extend our recipe to manage an Apache virtual host:

package ‘apache2‘ do
  action :install
end

template ‘/etc/apache2/sites-available/mysite.conf‘ do
  source ‘mysite.conf.erb‘
  owner ‘root‘
  group ‘root‘
  mode ‘0644‘
  notifies :restart, ‘service[apache2]‘
end

service ‘apache2‘ do
  action [:enable, :start]
end

This updated recipe adds an Apache config file using the template resource. The source property specifies an ERB template file to render. We‘ll create this template in a bit.

The notifies property instructs Chef to restart the Apache service if the config file changes. This ensures Apache picks up the new configuration.

To provide the content for our template, create a file named mysite.conf.erb in the templates/default directory:

<VirtualHost *:80>
  ServerName <%= node[‘hostname‘] %>
  ServerAdmin webmaster@<%= node[‘fqdn‘] %>
  DocumentRoot /var/www/html
  ErrorLog ${APACHE_LOG_DIR}/error.log
</VirtualHost>

This template uses ERB tags (<%= ... %>) to embed Ruby expressions. Here we‘re using node attributes to dynamically set the ServerName and ServerAdmin directives based on the system‘s hostname and fully-qualified domain name.

After saving the template, re-run the chef-client command to apply the updated recipe.

Congrats, you just used Chef to automate the configuration of a basic Apache web server! There are countless other resources you can use to manage things like users, cron jobs, networking, and more. Be sure to check out the Chef resource reference to explore all the possibilities.

Cookbook Dependencies with Berkshelf

So far we‘ve written all our Chef code from scratch. However, there‘s no need to reinvent the wheel for common infrastructure tasks. The Chef Supermarket is a public repository of community cookbooks you can use as building blocks.

To use community cookbooks, we need to declare them as dependencies of our cookbook. We do this using Berkshelf.

Open the metadata.rb file and add a line like:

depends ‘mysql‘, ‘~> 8.0‘

This declares our cookbook depends on version 8.x of the mysql community cookbook.

To install the dependencies, run:

berks install

This fetches the mysql cookbook and its dependencies and places them in the ~/.berkshelf/cookbooks directory. To upload the cookbooks to a Chef server, run:

berks upload

With Berkshelf, it‘s easy to leverage high-quality, community-maintained cookbooks to turbocharge your Chef development.

Attributes and Data Bags

We‘ve seen a couple ways to make our recipes dynamic and adaptable. ERB templates allow embedding Ruby expressions to interpolate values. The `node` object provides access to system data collected by Ohai – things like hostnames, IP addresses, platform details, and more.

Chef also provides two constructs for externalizing data from your recipes: attributes and data bags.

Attributes are key-value pairs associated with a node. They can be used to override default settings, specify node-specific values, or store data for use in recipes. To define attributes, create a file named attributes/default.rb in your cookbook:

default[‘my_webserver‘][‘port‘] = 80
default[‘my_webserver‘][‘document_root‘] = ‘/var/www/html‘

To use these attributes in a recipe:

template "#{node[‘my_webserver‘][‘document_root‘]}/index.html" do
  source ‘index.html.erb‘
  owner ‘root‘
  group ‘root‘  
  mode ‘0644‘
end

Data bags are global objects for storing arbitrary JSON data. They‘re useful for things like user credentials, service API keys, and other sensitive data you don‘t want to commit to version control.

To create a data bag item named "secrets":

mkdir data_bags/secrets
echo ‘{"api_key": "abc123"}‘ > data_bags/secrets/my_api_key.json

To read a data bag item in a recipe:

my_secret = data_bag_item(‘secrets‘, ‘my_api_key‘)
api_key = my_secret[‘api_key‘]

Attributes and data bags provide a powerful way to make your recipes flexible and keep credentials secure.

Roles, Environments, and Run Lists

As your infrastructure evolves, you‘ll likely have multiple types of servers – web servers, database servers, message queues, etc. Chef provides several constructs for organizing and classifying your nodes.

A role is a way to define certain patterns and processes that exist across nodes in your infrastructure. For example, you might have a "webserver" role that specifies the common configuration for your frontend web nodes.

Here‘s an example role file roles/webserver.rb:

name "webserver"
description "Web server role"
run_list "recipe[my_webserver]"
default_attributes({
  "my_webserver" => {
    "port" => 80
  }
})

This role uses the my_webserver recipe and sets a default port attribute.

An environment is a way to map your Chef configuration to your development workflow. You might have "dev", "staging", and "production" environments, each with different configuration values.

Here‘s an example environments/dev.rb file:

name "dev"
description "The development environment"
default_attributes({
  "my_webserver" => {
    "port" => 8080
  }
})

A run-list defines which recipes to run, and the order in which to run them, for a node. You can set a node‘s run-list when bootstrapping it with knife, or update it later using knife node run_list set.

For example, to bootstrap a new node with the "webserver" role:

knife bootstrap NODE_IP --ssh-user USER --node-name my-webserver --run-list ‘role[webserver]‘

Roles, environments, and run-lists give you fine-grained control over how Chef configures your nodes.

Putting It All Together

We‘ve covered a lot of ground in this guide! You‘ve learned what infrastructure as code is and why it matters. You‘ve seen how to write flexible, reusable Chef recipes using resources, templates, attributes, and more. And you‘ve discovered how to organize your infrastructure with roles, environments, and run-lists.

As next steps, I recommend:

1. Reading the official Chef documentation to go deeper on topics we‘ve introduced
2. Exploring the Chef Supermarket to find useful community cookbooks
3. Writing more cookbooks to automate your own infrastructure
4. Integrating Chef into your CI/CD pipeline to enable continuous delivery

Infrastructure as code takes practice, but it‘s worth the investment. By codifying your configuration, you can make your infrastructure more consistent, reliable, and scalable. Happy automating!