Use Serverspec for test-driven development of infrastructure code
Created by Sushant Jagdale (AWS)
Environment: PoC or pilot | Technologies: DevOps; Infrastructure; Hybrid cloud | AWS services: Amazon EC2; AWS CodeBuild; AWS CodeDeploy |
Summary
This pattern shows you how to use Serverspec
Serverspec helps with refactoring infrastructure code. With Serverspec, you can write RSpec tests to check installation of various packages and software, run commands, check for running processes and ports, check file permission settings, and so forth. Serverspec checks whether your servers are configured correctly. You install only Ruby on your servers. You don't need to install any agent software.
Test-driven infrastructure provides the following benefits:
Cross-platform testing
Validation of expectations
Confidence in your automation
Infrastructure consistency and stability
Fail early
You can use this pattern to run Serverspec unit tests for Apache software and check file permission settings during Amazon Machine Image (AMI) creation. An AMI will be created only if all the test cases pass. Serverspec will perform following tests:
Apache process is running.
Apache port is running.
Apache configuration files and directories exist at certain locations, and so forth.
File permissions are correctly configured.
Prerequisites and limitations
Prerequisites
An active AWS account
AWS CodeBuild
AWS CodeCommit
AWS CodePipeline
A virtual private cloud (VPC) with a public subnet
Installation of AWS Command Line Interface (AWS CLI) and Git
Product versions
HashiCorp Packer version: 1.6.6
Ruby version: 2.5.1 and later
AWS CLI version: 1.18.185
Architecture
Target architecture
![Developer workflow diagram showing CodeCommit, Event, CodePipeline, AWS CodeBuild, and AMI creation.](/images/prescriptive-guidance/latest/patterns/images/pattern-img/8092a64f-857c-4baa-b41c-a32c9098c4cf/images/eae48272-8e43-4e6c-aad8-e4fefe191fee.png)
When you push the code to the CodeCommit repository, an Amazon CloudWatch Events event engages the CodePipeline. In the first stage of the pipeline, the code is fetched from CodeCommit.
The second pipeline stage runs CodeBuild, which validates and builds the Packer template.
As a part of the Packer build provisioner, Packer installs Apache and Ruby software. Then the provisioner calls a shell script that uses Serverspec to unit test the Apache process, port, files, and directories. The Packer post-processor writes a JavaScript Object Notation (JSON) file with a list of all the artifacts produced by Packer during a run
Finally, an Amazon Elastic Compute Cloud (Amazon EC2) instance is created using the AMI ID produced by Packer.
Tools
AWS CLI – Amazon Command Line Interface (AWS CLI) is an open source tool for interacting with AWS services using commands in your command line shell.
Amazon CloudWatch Events – Amazon CloudWatch Events delivers a near-real-time stream of system events that describe changes in Amazon Web Services (AWS) resources.
AWS CodeBuild – AWS CodeBuild is a fully managed build service in the cloud. CodeBuild compiles your source code, runs unit tests, and produces artifacts that are ready to deploy.
AWS CodeCommit – AWS CodeCommit is a version control service hosted by Amazon Web Services. You can use CodeCommit to privately store and manage assets (such as documents, source code, and binary files) in the cloud.
AWS CodePipeline – AWS CodePipeline is a continuous delivery service you can use to model, visualize, and automate the steps required to release your software. You can quickly model and configure the different stages of a software release process.
HashiCorp Packer
– HashiCorp Packer is a tool for automating the creation of identical machine images from a single source configuration. Serverspec
– Serverspec runs RSpec tests to check server configuration. Serverspec uses Ruby, and you don't need to install agent software.
Code
The code is attached. The code uses the following structure, with three directories and eight files.
├── amazon-linux_packer-template.json (Packer template) ├── buildspec.yaml (CodeBuild .yaml file) ├── pipeline.yaml (AWS CloudFormation template to automate CodePipeline) ├── rspec_tests (RSpec required files and spec) │ ├── Gem-file │ ├── Rakefile │ └── spec │ ├── apache_spec.rb │ └── spec_helper.rb └── scripts └── rspec.sh (Installation of Ruby and initiation of RSpec)
Epics
Task | Description | Skills required |
---|---|---|
Create an IAM user. | Create an AWS Identity and Access Management (IAM) user with programmatic and console access. For more information, see the AWS documentation. | Developer, Systems administrator, DevOps engineer |
Configure AWS credentials. | On your local computer or in your environment, configure AWS credentials for the IAM user. For instructions, see the AWS documentation. | Developer, Systems administrator, DevOps engineer |
Test your credentials. | To validate the configured credentials, run the following command.
| Developer, Systems administrator, DevOps engineer |
Task | Description | Skills required |
---|---|---|
Create a CodeCommit repository. | To create a CodeCommit repository, run the following command.
| Developer, Systems administrator, DevOps engineer |
Write RSpec tests. | Create RSpec test cases for your infrastructure. For more information, see the Additional information section. | Developer, DevOps engineer |
Push code to the CodeCommit repository. | To push the attached code to the CodeCommit repository, run the following commands.
| Developer, Systems administrator, DevOps engineer |
Create the pipeline. | To create the pipeline, run the AWS CLI command that is in the Additional information section. | Developer, Systems administrator, DevOps engineer |
Start the pipeline. | Commit code to the CodeCommit repository. Any commit to the repository will initiate the pipeline. | Developer, Systems administrator, DevOps engineer |
Test the Apache URL. | To test the AMI installation, use the following URL.
The page will show a "Hello from Apache" message. | Developer, Systems administrator, DevOps engineer |
Related resources
Introduction to ServerSpec: What is Serverspec and how do we use it at Stelligent?
(external blog post) Test-driven development of infrastructure code
(external blog post) Image creation and testing with HashiCorp Packer and ServerSpec
(external article)
Additional information
Write RSpec tests
The RSpec test for this pattern is located at <repository folder>/rspec_tests/spec/apache_spec.rb
.
require 'spec_helper' describe service('httpd') do it { should be_enabled } it { should be_running } end describe port(80) do it { should be_listening } end describe file('/etc/httpd/conf/httpd.conf') do it { should exist } it { should be_owned_by 'root' } it { should contain 'ServerName www.example.com' } end describe file('/etc/httpd/conf/httpd.conf') do its(:content) { should match /ServerName www.example.com/ } end describe file('/var/www/html/hello.html') do it { should exist } it { should be_owned_by 'ec2-user' } end describe file('/var/log/httpd') do it { should be_directory } end describe file('/etc/sudoers') do it { should be_mode 440 } end describe group('root') do it { should have_gid 0 } end
You can add your own tests to the /spec
directory.
Create the pipeline
aws cloudformation create-stack --stack-name myteststack --template-body file://pipeline.yaml --parameters ParameterKey=RepositoryName,ParameterValue=<provide repository-name> ParameterKey=ApplicationName,ParameterValue=<provide application-name> ParameterKey=SecurityGroupId,ParameterValue=<provide SecurityGroupId> ParameterKey=VpcId,ParameterValue=<provide VpcId> ParameterKey=SubnetId,ParameterValue=<provide SubnetId> ParameterKey=Region,ParameterValue=<provide Region> ParameterKey=Keypair,ParameterValue=<provide Keypair> ParameterKey=AccountId,ParameterValue=<provide AccountId> --capabilities CAPABILITY_NAMED_IAM
Parameter details
repository-name
– The name of the AWS CodeCommit repository
application-name
– The Amazon Resource Name (ARNs) are linked with ApplicationName
; provide any name
SecurityGroupId
– Any security group ID from your AWS account that has port 80 open
VpcId
– The ID of your VPC
SubnetId
– The ID of a public subnet in your VPC
Region
– The AWS Region where you are running this pattern
Keypair
– The Secure Shell (SSH) key name to log in to the EC2 instance
AccountId
– Your AWS account ID
You can also create a CodePipeline pipeline by using the AWS Management Console and passing the same parameters that are in the previous command line.
Attachments
To access additional content that is associated with this document, unzip the following file: attachment.zip