Treating infrastructure as code refers to writing code, via any descriptive language, to manage server provisioning and configuration process in addition to deployment of actual application code. It involves adopting automated testing and related practices that are already used in modern application development, such as version control, integration testing, small deployments, use of design patterns etc.
Following those practices and rules means writing code to provision and manage the configuration of your servers. By doing that you will gain safe, repeatable and automated processes.
Chef as Framework
As software development has matured, frameworks have emerged with the aim of reducing development time and allowing developers to concentrate on rapid delivery of software that meets requirements. Software frameworks provide conventions with the goal to encourage consistency across a team and facilitate sustainable development of a project.
One of the popular frameworks for managing infrastructure provisioning and configuration is Chef. It provides an extensive library of primitives for managing every resource that is used in the process of building and maintaining web infrastructure. Powered by a Ruby-based domain specific language (DSL), it provides a consistent abstraction layer that allows developers and system administrators to design and build scalable environments with ease.
Chef Components
The Chef process consists of three core components that interact with one another: Chef server, actual servers called nodes, and Chef workstation.
Chef Server
The Chef server is as a hub for configuration data. It stores cookbooks, the policies that are applied to nodes, and metadata that describes each node managed by Chef.
Nodes
Nodes use a tool called Chef client to ask the Chef server for configuration details and then applies them to itself. This process of applying changes on nodes is called a Chef run.
Cookbooks are made of one or more recipes that will perform automated steps, called actions, such as installing and configuring applications or creating files. All data needed in this process is downloaded from the Chef server as the first step in a Chef run.
Note that the Chef client is installed within the bootstrap process, when a node is created and registered on a Chef server.
Each node can have one or more roles. A role defines a set of attributes (node specific settings) and a list of recipes for a node. It can be reused by multiple nodes, so you can have a cluster of nodes with the same role in the system. The computed list of recipes associated with a node via role or as dependency of other recipes is called a run list, and it is executed in the same order it’s defined.
Chef Workstation
The Chef workstation or Chef repository, shortly chef-repo, is the project structure of a Chef-managed project and it is used on a developer workstation. All Chef components are defined in it: cookbooks, environments, roles and a test suite. It is a good and common practise to keep the chef-repo in a version control system and manage the same way you would manage the source code of your application.
The exact directory structure of a chef-repo varies. Some people prefer to keep all their cookbooks in a single chef-repo. Others prefer to use a separate repository for each cookbook. In both situations your main chef-repo is the place from where you manage your infrastructure, and it should contain information about all of its parts.
An Overview of Chef Components
Here is a short summary of main parts of Chef architecture:
- Chef Server: Centralized server that holds all of your nodes’ configuration. It can be self-hosted or hosted by Chef (the company).
- Node: Hosts to which recipes and roles are applied during Chef client run. The primary features of a node are its attributes and run list.
- Cookbooks: Contain all resources and instructions that you need to configure your nodes. These can be reused across different run lists. Cookbooks typically consist of many recipes.
- Recipe: The fundamental part of Chef, it is a collection of resources that are executed in the order to configure a node.
- Resource: A cross platform abstraction of configurable parts of a node. For example these could be users, packages, files or directories.
- Attributes – Represent node settings, for example hostname, versions of programming languages to install, database server etc.
- Data bags: Contain globally available data used by nodes and roles.
- Chef Client: Does all work on behalf of a node, where it executes recipes to configure and install software.
- Chef Repository: The place where cookbooks, roles, configuration files, and other artifacts live.
- Chef Solo: A command line tool that allows you to run Chef cookbooks without an actual Chef server. It is an open source version of the Chef client.
- Knife: A tool used by engineers to upload configuration changes to the Chef server.
- Ohai: A tool for collecting data about your operating system, used to provide system attributes used by Chef client during Chef run.
- Role: A way to group similar features of nodes, for easier management.
You can find more detailed introductions, along with best practices and tutorials on the learning section of Chef’s website.
Chef Development Environment
A Chef repository is a location from which you will do the most of your work, including:
- Developing cookbooks and recipes;
- Uploading items to the Chef server;
- Keeping the Chef repository synchronized with version source control;
- Managing configuration options, roles and environments, and
- Interacting with existing and provisioning new nodes.
The role of a development workstation in the Chef stack is to synchronize your Chef repository with Chef server and all nodes it interacts with.
A standard set of tools used to manage Chef includes two important command line tools. One is knife, the command line tool for interacting with Chef server and synchronizing nodes. The other is chef, for working with the Chef repository components and managing the embedded Ruby environment used by Chef itself.
Aside from the built-in tools, there is a range of supporting tools widely adopted in the Chef community. Some of them are:
- Berkshelf – A cookbook dependency manager.
- Test Kitchen – An integration testing framework.
- ChefSpec – A unit testing framework.
- Foodcritic – A linting tool for performing static code analysis in cookbooks.
Installing the Chef Development Kit
To make a development environment setup easy, the Chef community has created the Chef Development Kit (ChefDK). The aim of this project is to bring all widely adopted tools in one easily installable package, with support for many different platforms.
To start using Chef and its supporting tools we need to install ChefDK first. To do that, go to ChefDK downloads page and download the right package for your platform. After downloading it, simply follow the standard installation steps for your platform.
Note that ChefDK on UNIX and Linux systems will be installed to /opt/chefdk/.
Results of installation can be verified by running Chef’s built in command:
$ chef verify
Running verification for component 'chef-dk'
'...'
Running verification for component 'chefspec'
---------------------------------------------
Verification of component 'chef-dk' succeeded.
'...'
Verification of component 'chefspec' succeeded.
You can consult the official documentation for more detailed instructions about this process.
If installation passed without any issues and all components are in place, you are ready to start using Chef to manage your infrastructure. You have now all required tools to create and test your recipes, upload them to the Chef server and provision nodes.
Wrapping up
By using configuration management tools like Chef to manage your infrastructure you can take the best practices from application development and apply them to development and scaling of your server infrastructure. In the following articles in this series we will explore how to apply test-driven development to infrastructure.