Migrate ServerSpec to InSpec - Part 1

From Bonus Bits
Jump to: navigation, search

Purpose

This article gives the steps to migrate from ServerSpec to InSpec Integration tests in a Chef Cookbook. The steps are split into several small parts. Part 1 covers creating a simple Chef Cookbook with ServerSpec tests. We'll throw together a simple Nginx/PHP cookbook running on Amazon Linux Docker Image.


Basic Project Setup

  1. Create a Git repo, I'll be using Github in my examples.
    1. I created a repository named example_serverspec_to_inspec on the bonusbits Github organization
    2. I then created a branch named 01_serverspec to store the finished product of Part 1 as reference
  2. Clone Locally
    git clone https://github.com/bonusbits/example_serverspec_to_inspec.git
    


Generate Cookbook Skeleton

  1. cd example_serverspec_to_inspec
    
  2. chef generate cookbook .
    
  3. Delete unneeded content
    rm -rf test/smoke
    
    rm -rf spec
    
    rm -rf .delivery
    


Cookbook Files

.gitignore

.vagrant
*~
*#
.#*
\#*#
.*.sw[a-z]
*.un~

# Bundler
Gemfile.lock
bin/*
.bundle/*

# test kitchen
.kitchen/
.kitchen.local.yml

# Chef
Berksfile.lock
.zero-knife.rb
Policyfile.lock.json
.idea

.rubocop.yml

AllCops:
  Exclude:
    - 'berks-cookbooks/**/*'
    - '.idea/**/*'
    - '.git/**/*'
    - '.kitchen/**/*'
    - '*.lock'
    - 'vendor/**/*'

Documentation:
  Enabled: false

Metrics/LineLength:
  Max: 200

Metrics/BlockLength:
  Enabled: false

Metrics/MethodLength:
  Max: 60
  CountComments: false
  Enabled: true

Style/Encoding:
  Enabled: false

Style/SignalException:
  EnforcedStyle: only_raise

Style/EmptyLiteral:
  Enabled: false

CyclomaticComplexity:
  Enabled: false

AlignParameters:
  Enabled: false

Encoding:
  Enabled: false

Style/FrozenStringLiteralComment:
  Enabled: false

Style/TrailingCommaInLiteral:
  Enabled: false

.kitchen.yml

---
driver:
  name: docker
  use_sudo: false
  privileged: true
  forward: 80
  driver_config:
    ssl_verify_mode: ":verify_none"

provisioner:
  name: chef_zero
  always_update_cookbooks: true
  require_chef_omnibus: 12.19.36

platforms:
  - name: amazon-docker
    driver_config:
      image: amazonlinux:latest
      platform: rhel

suites:
  - name: default
    run_list:
      - recipe[example_serverspec_to_inspec::default]
    attributes:

Rakefile

# Rubocop and Foodcritic
namespace :style do
  require 'foodcritic'
  desc 'FoodCritic'
  FoodCritic::Rake::LintTask.new(:chef) do |task|
    task.options = {
      fail_tags: ['correctness'],
      chef_version: '12.19.36',
      tags: %w(~FC001 ~FC019 ~FC016 ~FC039)
    }
  end

  require 'rubocop/rake_task'
  desc 'RuboCop'
  RuboCop::RakeTask.new(:ruby) do |task|
    task.options = ['--display-cop-names']
  end
end

# Test Kitchen
desc 'Kitchen'
task :integration do
  require 'kitchen/rake_tasks'
  Kitchen.logger = Kitchen.default_file_logger
  kitchen_loader = Kitchen::Loader::YAML.new(local_config: '.kitchen.yml')
  Kitchen::Config.new(loader: kitchen_loader, log_level: :info).instances.each do |instance|
    instance.test(:always)
  end
end

desc 'Foodcritic & Rubocop'
task default: %w(style:chef style:ruby)

desc 'Foodcritic & Rubocop'
task style_tasks: %w(style:chef style:ruby)

desc 'Circle CI Tasks'
task circleci: %w(style:chef style:ruby integration:docker)

Gemfile

source 'https://rubygems.org'

gem 'berkshelf', '~> 5.6'
gem 'rake', '~> 12.0.0'

group :style do
  gem 'chef', '12.19.36'
  gem 'foodcritic', '~> 10.2'
  gem 'rainbow', '~> 2.2.1'
  gem 'rubocop', '~> 0.47.1'
end

group :integration do
  gem 'chef-zero', '~> 5.3'
  gem 'kitchen-docker', '~> 2.6.0'
  gem 'test-kitchen', '~> 1.16'
end

circle.yml

version: 2
jobs:
  build:
    machine: true
    services:
      - docker
    ruby:
      version: '2.3.1'
    timezone:
      America/Los_Angeles
    working_directory: ~/circulate
    steps:
      - checkout
      - run:
          name: Create Test Directory
          command: mkdir test-reports
      - run:
          name: Bundle Gems
          command: bundle check --path=vendor/bundle || bundle install --path=vendor/bundle --jobs=4 --retry=3
      - run:
          name: Run Rake Task
          command: bundle exec rake circleci --trace
      - store_test_results:
          path: test-reports/
notify:
  webhooks:
    - url: https://webhooks.gitter.im/e/00000000000000000

metadata.rb

name 'example_serverspec_to_inspec'
maintainer 'First Last'
maintainer_email 'first.last@domain.com'
license 'MIT'
description 'Demonstrate Migration Integration Tests from ServerSpec to InSpec'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version '1.0.0'
chef_version '~> 12.5' if respond_to?(:chef_version)
issues_url 'https://github.com/bonusbits/example_serverspec_to_inspec/issues'
source_url 'https://github.com/bonusbits/example_serverspec_to_inspec'

supports 'amazon'

recipe/default.rb

template '/etc/sysconfig/network' do
  source 'sysconfig.network.erb'
  owner 'root'
  group 'root'
  mode '0644'
  not_if { ::File.exist?('/etc/sysconfig/network') }
end

package %w(php70-fpm nginx)

ruby_block 'Set PHP FPM Ownership' do
  block do
    command = 'sed -i \'s/apache/nginx/g\' /etc/php-fpm-7.0.d/www.conf'
    command += ' && chown -R nginx:nginx /var/log/php-fpm'
    command += ' && chown -R nginx:nginx /var/lib/php/7.0'
    require 'open3'
    out, err, status = Open3.capture3(command)
    Chef::Log.warn("Open3: Standard Out (#{out})")
    Chef::Log.warn("Open3: Status (#{status})")
    raise "Open3: Status (#{err})" unless status.success?
  end
  not_if { ::File.readlines('/etc/php-fpm-7.0.d/www.conf').grep(/^user = nginx/).any? }
end

# Enable and Start Service
service 'php-fpm-7.0' do
  action [:enable, :start]
end

# Enable and Start Service
service 'nginx' do
  action [:enable, :start]
end

templates/default/sysconfig.network.erb

NETWORKING=yes
HOSTNAME=localhost.localdomain
NOZEROCONF=yes

test/integration/helpers/serverspec/spec_helper.rb

# Encoding: utf-8
require 'serverspec'

if (/cygwin|mswin|mingw|bccwin|wince|emx/ =~ RUBY_PLATFORM).nil?
  set :backend, :exec
  set :path, '/sbin:/usr/local/sbin:/bin:/usr/bin:$PATH'
else
  set :backend, :cmd
  set :os, family: 'windows'
end

test/integration/default/serverspec/nginx_spec.rb

require 'spec_helper'

describe 'Nginx' do
  it 'nginx installed' do
    expect(package('nginx')).to be_installed
  end

  it 'nginx service' do
    expect(service('nginx')).to be_enabled
    expect(service('nginx')).to be_running
  end
end

test/integration/default/serverspec/phpfpm_spec.rb

require 'spec_helper'

describe 'Php FPM' do
  it 'php-fpm installed' do
    expect(package('php70-fpm')).to be_installed
  end

  it 'php-fpm service' do
    expect(service('php-fpm-7.0')).to be_enabled
    expect(service('php-fpm-7.0')).to be_running
  end

  it 'nginx owns /var/log/php-fpm' do
    expect(file('/var/log/php-fpm')).to be_owned_by('nginx')
    expect(file('/var/log/php-fpm/7.0')).to be_owned_by('nginx')
  end

  it 'nginx owns /var/lib/php/7.0' do
    expect(file('/var/lib/php/7.0')).to be_grouped_into('nginx')
  end
end


Run ServerSpec Tests

Now that we have a basic cookbook using ServerSpec tests, the last step is to run the integration tests.

rake integration

OR

kitchen test

Example

-----> Cleaning up any prior instances of <default-amazon-docker>
-----> Destroying <default-amazon-docker>...
       Error response from daemon: Container e13d149ffb68f112504429290da9f2540c86d6a9f3fc8845a990b8bb23fde664 is not running
       Finished destroying <default-amazon-docker> (0m0.04s).
-----> Testing <default-amazon-docker>
-----> Creating <default-amazon-docker>...
       Sending build context to Docker daemon  466.9kB
       Step 1/16 : FROM amazonlinux:latest
        ---> 766ebb052d4f
       Step 2/16 : ENV container docker
        ---> Using cache
        ---> 9306967c7449
       Step 3/16 : RUN yum clean all
        ---> Using cache
        ---> 6729a55d63b3
       Step 4/16 : RUN yum install -y sudo openssh-server openssh-clients which curl
        ---> Using cache
        ---> d91055dade53
       Step 5/16 : RUN ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key -N ''
        ---> Using cache
        ---> c647407017ec
       Step 6/16 : RUN ssh-keygen -t dsa -f /etc/ssh/ssh_host_dsa_key -N ''
        ---> Using cache
        ---> dde4401a6122
       Step 7/16 : RUN if ! getent passwd kitchen; then                 useradd -d /home/kitchen -m -s /bin/bash -p '*' kitchen;               fi
        ---> Using cache
        ---> 43bc21da1946
       Step 8/16 : RUN echo "kitchen ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
        ---> Using cache
        ---> c13e2988ada5
       Step 9/16 : RUN echo "Defaults !requiretty" >> /etc/sudoers
        ---> Using cache
        ---> 4fabb287c4c5
       Step 10/16 : RUN mkdir -p /home/kitchen/.ssh
        ---> Using cache
        ---> fa81cf0825ed
       Step 11/16 : RUN chown -R kitchen /home/kitchen/.ssh
        ---> Using cache
        ---> 6742e97caba3
       Step 12/16 : RUN chmod 0700 /home/kitchen/.ssh
        ---> Using cache
        ---> 790ecdc0c8a1
       Step 13/16 : RUN touch /home/kitchen/.ssh/authorized_keys
        ---> Using cache
        ---> 682afe7e98ab
       Step 14/16 : RUN chown kitchen /home/kitchen/.ssh/authorized_keys
        ---> Using cache
        ---> fa4279bfb34f
       Step 15/16 : RUN chmod 0600 /home/kitchen/.ssh/authorized_keys
        ---> Using cache
        ---> 4ebe38dc87f3
       Step 16/16 : RUN echo ssh-rsa\ AAAAB3NzaC1yc2EAAAADAQABAAABAQDh0jPG\+f\+JLWB9ckvab0iAhiae/ICRQ0JbIOrtwEgGsLisWLPDGE81JxXR0UuwUKuoJ30F2Mih1YgcoE7qYAouBUSseNgDs3E5O8V1IVjSGGdXXkFJf56jU7dABGbavXH3tq0l1NnBtx6krJNO5xhlJU5QhgZlbu\+1r8/g1AkEIVbWYLjaPMciicmEQ\+j\+IUNTjuMBy4rWaE\+37MmE8QgD5XL/NDkEyhZ1u2vegKebvCEcX2IhNNUnyPkVXz2lqeCyrQX/MczeguX0rtxQbJyQ3WU1/O/u5YS\+na6Td16hJt/fGQzTZkUl\+8MYqS8xFNn\+hdlZ7Fd\+QgEvs/mTYc5B\ kitchen_docker_key >> /home/kitchen/.ssh/authorized_keys
        ---> Using cache
        ---> 4ad91eb3a6dd
       Successfully built 4ad91eb3a6dd
       d1a56632d8f4d744e0e2fd33a1656345bd3abf846da1e33922ae59cb2d887e1f
       0.0.0.0:32769
       Waiting for SSH service on localhost:32769, retrying in 3 seconds
       [SSH] Established
       Finished creating <default-amazon-docker> (0m3.99s).
-----> Converging <default-amazon-docker>...
       Preparing files for transfer
       Preparing dna.json
       Resolving cookbook dependencies with Berkshelf 5.6.4...
       Removing non-cookbook files before transfer
       Preparing validation.pem
       Preparing client.rb
-----> Installing Chef Omnibus (12.19.36)
       Downloading https://omnitruck.chef.io/install.sh to file /tmp/install.sh
       Trying curl...
       Download complete.
       el 6 x86_64
       Getting information for chef stable 12.19.36 for el...
       downloading https://omnitruck.chef.io/stable/chef/metadata?v=12.19.36&p=el&pv=6&m=x86_64
         to file /tmp/install.sh.31/metadata.txt
       trying curl...
       sha1	323e88bd64b166a823087d0f50a949506d0fa6b5
       sha256	89e8e6e9aebe95bb31e9514052a8926f61d82067092ca3bc976b0bd223710c81
       url	https://packages.chef.io/files/stable/chef/12.19.36/el/6/chef-12.19.36-1.el6.x86_64.rpm
       version	12.19.36
       downloaded metadata file looks valid...
       downloading https://packages.chef.io/files/stable/chef/12.19.36/el/6/chef-12.19.36-1.el6.x86_64.rpm
         to file /tmp/install.sh.31/chef-12.19.36-1.el6.x86_64.rpm
       trying curl...
       Comparing checksum with sha256sum...
       Installing chef 12.19.36
       installing with rpm...
       warning: /tmp/install.sh.31/chef-12.19.36-1.el6.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 83ef826a: NOKEY
       Preparing...                          ################################# [100%]
       Updating / installing...
          1:chef-12.19.36-1.el6              ################################# [100%]
       Thank you for installing Chef!
       Transferring files to <default-amazon-docker>
       [2017-04-24T04:02:57+00:00] INFO: Forking chef instance to converge...
       Starting Chef Client, version 12.19.36
       [2017-04-24T04:02:57+00:00] INFO: *** Chef 12.19.36 ***
       [2017-04-24T04:02:57+00:00] INFO: Platform: x86_64-linux
       [2017-04-24T04:02:57+00:00] INFO: Chef-client pid: 151
       Creating a new client identity for default-amazon-docker using the validator key.
       [2017-04-24T04:02:58+00:00] INFO: Client key /tmp/kitchen/client.pem is not present - registering
       [2017-04-24T04:02:58+00:00] INFO: HTTP Request Returned 404 Not Found: Object not found: chefzero://localhost:8889/nodes/default-amazon-docker
       [2017-04-24T04:02:58+00:00] INFO: Setting the run_list to ["recipe[example_serverspec_to_inspec::default]"] from CLI options
       [2017-04-24T04:02:58+00:00] INFO: Run List is [recipe[example_serverspec_to_inspec::default]]
       [2017-04-24T04:02:58+00:00] INFO: Run List expands to [example_serverspec_to_inspec::default]
       [2017-04-24T04:02:58+00:00] INFO: Starting Chef Run for default-amazon-docker
       [2017-04-24T04:02:58+00:00] INFO: Running start handlers
       [2017-04-24T04:02:58+00:00] INFO: Start handlers complete.
       [2017-04-24T04:02:58+00:00] INFO: HTTP Request Returned 404 Not Found: Object not found:
       resolving cookbooks for run list: ["example_serverspec_to_inspec::default"]
       [2017-04-24T04:02:58+00:00] INFO: Loading cookbooks [example_serverspec_to_inspec@1.0.0]
       Synchronizing Cookbooks:
       [2017-04-24T04:02:58+00:00] INFO: Storing updated cookbooks/example_serverspec_to_inspec/recipes/default.rb in the cache.
       [2017-04-24T04:02:58+00:00] INFO: Storing updated cookbooks/example_serverspec_to_inspec/metadata.json in the cache.
       [2017-04-24T04:02:58+00:00] INFO: Storing updated cookbooks/example_serverspec_to_inspec/README.md in the cache.
       [2017-04-24T04:02:58+00:00] INFO: Storing updated cookbooks/example_serverspec_to_inspec/templates/default/sysconfig.network.erb in the cache.
         - example_serverspec_to_inspec (1.0.0)
       Installing Cookbook Gems:
       Compiling Cookbooks...
       Converging 5 resources
       Recipe: example_serverspec_to_inspec::default
         * template[/etc/sysconfig/network] action create[2017-04-24T04:02:58+00:00] INFO: Processing template[/etc/sysconfig/network] action create (example_serverspec_to_inspec::default line 7)
       [2017-04-24T04:02:58+00:00] INFO: template[/etc/sysconfig/network] created file /etc/sysconfig/network

           - create new file /etc/sysconfig/network[2017-04-24T04:02:58+00:00] INFO: template[/etc/sysconfig/network] updated file contents /etc/sysconfig/network

           - update content in file /etc/sysconfig/network from none to 9d2f64
           --- /etc/sysconfig/network	2017-04-24 04:02:58.417614163 +0000
           +++ /etc/sysconfig/.chef-network20170424-151-1rpniw3	2017-04-24 04:02:58.417614163 +0000
           @@ -1 +1,4 @@
           +NETWORKING=yes
           +HOSTNAME=localhost.localdomain
           +NOZEROCONF=yes[2017-04-24T04:02:58+00:00] INFO: template[/etc/sysconfig/network] owner changed to 0
       [2017-04-24T04:02:58+00:00] INFO: template[/etc/sysconfig/network] group changed to 0
       [2017-04-24T04:02:58+00:00] INFO: template[/etc/sysconfig/network] mode changed to 644

           - change mode from '' to '0644'
           - change owner from '' to 'root'
           - change group from '' to 'root'
         * yum_package[php70-fpm, nginx] action install[2017-04-24T04:02:58+00:00] INFO: Processing yum_package[php70-fpm, nginx] action install (example_serverspec_to_inspec::default line 15)
       [2017-04-24T04:03:03+00:00] INFO: yum_package[php70-fpm, nginx] installing php70-fpm-7.0.16-1.21.amzn1 from amzn-main repository nginx-1.10.2-1.30.amzn1 from amzn-main repository
       [2017-04-24T04:03:08+00:00] INFO: yum_package[php70-fpm, nginx] installed ["php70-fpm", "nginx"] at ["7.0.16-1.21.amzn1", "1.10.2-1.30.amzn1"]

           - install version 7.0.16-1.21.amzn1 of package php70-fpm
           - install version 1.10.2-1.30.amzn1 of package nginx
         * ruby_block[Set PHP FPM Ownership] action run[2017-04-24T04:03:08+00:00] INFO: Processing ruby_block[Set PHP FPM Ownership] action run (example_serverspec_to_inspec::default line 17)
       [2017-04-24T04:03:08+00:00] WARN: Open3: Standard Out ()
       [2017-04-24T04:03:08+00:00] WARN: Open3: Status (pid 433 exit 0)
       [2017-04-24T04:03:08+00:00] INFO: ruby_block[Set PHP FPM Ownership] called

           - execute the ruby block Set PHP FPM Ownership
         * service[php-fpm-7.0] action enable[2017-04-24T04:03:08+00:00] INFO: Processing service[php-fpm-7.0] action enable (example_serverspec_to_inspec::default line 32)
       [2017-04-24T04:03:08+00:00] INFO: service[php-fpm-7.0] enabled

           - enable service service[php-fpm-7.0]
         * service[php-fpm-7.0] action start[2017-04-24T04:03:08+00:00] INFO: Processing service[php-fpm-7.0] action start (example_serverspec_to_inspec::default line 32)
       [2017-04-24T04:03:08+00:00] INFO: service[php-fpm-7.0] started

           - start service service[php-fpm-7.0]
         * service[nginx] action enable[2017-04-24T04:03:08+00:00] INFO: Processing service[nginx] action enable (example_serverspec_to_inspec::default line 37)
       [2017-04-24T04:03:08+00:00] INFO: service[nginx] enabled

           - enable service service[nginx]
         * service[nginx] action start[2017-04-24T04:03:08+00:00] INFO: Processing service[nginx] action start (example_serverspec_to_inspec::default line 37)
       [2017-04-24T04:03:09+00:00] INFO: service[nginx] started

           - start service service[nginx]
       [2017-04-24T04:03:09+00:00] INFO: Chef Run complete in 10.73773455 seconds

       Running handlers:
       [2017-04-24T04:03:09+00:00] INFO: Running report handlers
       Running handlers complete
       [2017-04-24T04:03:09+00:00] INFO: Report handlers complete
       Chef Client finished, 7/7 resources updated in 12 seconds
       Finished converging <default-amazon-docker> (0m29.30s).
-----> Setting up <default-amazon-docker>...
       Finished setting up <default-amazon-docker> (0m0.00s).
-----> Verifying <default-amazon-docker>...
       Preparing files for transfer
-----> Installing Busser (busser)
Fetching: thor-0.19.0.gem (100%)
       Successfully installed thor-0.19.0
Fetching: busser-0.7.1.gem (100%)
       Successfully installed busser-0.7.1
       2 gems installed
       Installing Busser plugins: busser-serverspec
       Plugin serverspec installed (version 0.5.10)
-----> Running postinstall for serverspec plugin
       Suite path directory /tmp/verifier/suites does not exist, skipping.
       Transferring files to <default-amazon-docker>
-----> Running serverspec test suite
-----> Installing Serverspec..
Fetching: diff-lcs-1.3.gem (100%)
Fetching: rspec-expectations-3.5.0.gem (100%)
Fetching: rspec-mocks-3.5.0.gem (100%)
Fetching: rspec-3.5.0.gem (100%)
Fetching: rspec-its-1.2.0.gem (100%)
Fetching: multi_json-1.12.1.gem (100%)
Fetching: net-ssh-4.1.0.gem (100%)
Fetching: net-scp-1.2.1.gem (100%)
Fetching: net-telnet-0.1.1.gem (100%)
Fetching: sfl-2.3.gem (100%)
Fetching: specinfra-2.67.8.gem (100%)
Fetching: serverspec-2.38.0.gem (100%)
-----> serverspec installed (version 2.38.0)
       /opt/chef/embedded/bin/ruby -I/tmp/verifier/suites/serverspec -I/tmp/verifier/gems/gems/rspec-support-3.5.0/lib:/tmp/verifier/gems/gems/rspec-core-3.5.4/lib /opt/chef/embedded/bin/rspec --pattern /tmp/verifier/suites/serverspec/\*\*/\*_spec.rb --color --format documentation --default-path /tmp/verifier/suites/serverspec

       Nginx
         nginx installed
         nginx service

       Php FPM
         php-fpm installed
         php-fpm service
         nginx owns /var/log/php-fpm
         nginx owns /var/lib/php/7.0

       Finished in 0.21613 seconds (files took 0.30629 seconds to load)
       6 examples, 0 failures

       Finished verifying <default-amazon-docker> (0m6.42s).
       Finished testing <default-amazon-docker> (0m39.77s).
-----> Destroying <default-amazon-docker>...
       PID                 USER                TIME                COMMAND
       3143                root                0:00                /usr/sbin/sshd -D -o UseDNS=no -o UsePAM=no -o PasswordAuthentication=yes -o UsePrivilegeSeparation=no -o PidFile=/tmp/sshd.pid
       3174                root                0:00                sshd: kitchen@pts/0
       3666                root                0:00                {php-fpm-7.0} php-fpm: master process (/etc/php-fpm.conf)
       3667                499                 0:00                {php-fpm-7.0} php-fpm: pool www
       3668                499                 0:00                {php-fpm-7.0} php-fpm: pool www
       3669                499                 0:00                {php-fpm-7.0} php-fpm: pool www
       3670                499                 0:00                {php-fpm-7.0} php-fpm: pool www
       3671                499                 0:00                {php-fpm-7.0} php-fpm: pool www
       3742                root                0:00                nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
       3743                499                 0:00                nginx: worker process
       3744                499                 0:00                nginx: worker process
       3745                499                 0:00                nginx: worker process
       3746                499                 0:00                nginx: worker process
       d1a56632d8f4d744e0e2fd33a1656345bd3abf846da1e33922ae59cb2d887e1f
       d1a56632d8f4d744e0e2fd33a1656345bd3abf846da1e33922ae59cb2d887e1f
       Finished destroying <default-amazon-docker> (0m1.29s).


Reference Github Link


Walkthrough Video


Related Articles