Migrate ServerSpec to InSpec - Part 2

From Bonus Bits
Jump to: navigation, search

Purpose

This article gives the steps to migrate from ServerSpec to InSpec Integration tests in a Chef Cookbook. The steps are split into several small parts. Part 2 walks through converting the ServerSpec local tests to InSpec local tests.

Changes to Cookbook

Remove Helper Folder

First, let's remove the spec helper that essentially required the serverspec gem.

rm -rf test/integration/helpers

Rename serverspec Folder to inspec

Next, let's rename the serverspec folder to inspec.

mv test/integration/default/serverspec test/integration/default/inspec

Remove spec_helper from each Spec Test File

Remove the first line of each of the two integration test specification files.

Icon-Tip-Square-Green.png Also, with InSpec they don't have to be named blah_spec.rb anymore.

test/integration/default/serverspec/nginx_spec.rb

describe 'Nginx' do
  it 'nginx installed' do
    expect(package('nginx')).to be_installed
  end

  it 'nginx service' do
    expect(service('nginx')).to be_enabled
    expect(service('nginx')).to be_running
  end
end

test/integration/default/serverspec/phpfpm_spec.rb

describe 'Php FPM' do
  it 'php-fpm installed' do
    expect(package('php70-fpm')).to be_installed
  end

  it 'php-fpm service' do
    expect(service('php-fpm-7.0')).to be_enabled
    expect(service('php-fpm-7.0')).to be_running
  end

  it 'nginx owns /var/log/php-fpm' do
    expect(file('/var/log/php-fpm')).to be_owned_by('nginx')
    expect(file('/var/log/php-fpm/7.0')).to be_owned_by('nginx')
  end

  it 'nginx owns /var/lib/php/7.0' do
    expect(file('/var/lib/php/7.0')).to be_grouped_into('nginx')
  end
end

Add InSpec Gems to Gemfile

Now, let's add two Rubygems named inspec and kitchen-inspec to our Gemfile integration group:

source 'https://rubygems.org'

gem 'berkshelf', '~> 5.6'
gem 'rake', '~> 12.0.0'

group :style do
  gem 'chef', '12.19.36'
  gem 'foodcritic', '~> 10.2'
  gem 'rainbow', '~> 2.2.1'
  gem 'rubocop', '~> 0.47.1'
end

group :integration do
  gem 'chef-zero', '~> 5.3'
  gem 'inspec', '~> 1.19'
  gem 'kitchen-docker', '~> 2.6.0'
  gem 'kitchen-inspec', '~> 0.17'
  gem 'test-kitchen', '~> 1.16'
end

Update .kitchen.yml

Next, add verifier section to our kitchen configuration. The minimum we need is verifier: name: inspec. The default format is cli. For our CI CircleCI we can set format to junit and output to a file. Then in the CircleCi config setup a folder to write the artifact and point to it for CircleCi to pickup for it's Test Summary. This is not required but is a nice to have and makes it quicker to find results instead of digging through logs in the console.

---

verifier:
  name: inspec
  format: <%= ENV['CI'] ? 'junit' : 'cli' %>
  <% if ENV['CI'] %>
  output: "test-reports/%{platform}_%{suite}_inspec.xml"
  <% end %>

driver:
  name: docker
  use_sudo: false
  privileged: true
  forward: 80
  driver_config:
    ssl_verify_mode: ":verify_none"

provisioner:
  name: chef_zero
  always_update_cookbooks: true
  require_chef_omnibus: 12.19.36

platforms:
  - name: amazon-docker
    driver_config:
      image: amazonlinux:latest
      platform: rhel

suites:
  - name: default
    run_list:
      - recipe[example_serverspec_to_inspec::default]
    attributes:

Revision Cookbook Version in metadata.rb

We'll go ahead and bump the cookbook version to 1.1.0.

name 'example_serverspec_to_inspec'
maintainer 'First Last'
maintainer_email 'first.last@domain.com'
license 'MIT'
description 'Demonstrate Migration Integration Tests from ServerSpec to InSpec'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version '1.1.0'
chef_version '~> 12.5' if respond_to?(:chef_version)
issues_url 'https://github.com/bonusbits/example_serverspec_to_inspec/issues'
source_url 'https://github.com/bonusbits/example_serverspec_to_inspec'

supports 'amazon'


Run InSpec Tests

Now that we have a converted our ServerSpec tests to InSpec tests, the last step is to run the integration tests.

rake integration

OR

kitchen test

Example

# kitchen test
-----> Starting Kitchen (v1.16.0)
-----> Cleaning up any prior instances of <default-amazon-docker>
-----> Destroying <default-amazon-docker>...
       Finished destroying <default-amazon-docker> (0m0.00s).
-----> Testing <default-amazon-docker>
-----> Creating <default-amazon-docker>...
       Sending build context to Docker daemon  487.9kB
       Step 1/16 : FROM amazonlinux:latest
        ---> 766ebb052d4f
       Step 2/16 : ENV container docker
        ---> Using cache
        ---> 9306967c7449
       Step 3/16 : RUN yum clean all
        ---> Using cache
        ---> 6729a55d63b3
       Step 4/16 : RUN yum install -y sudo openssh-server openssh-clients which curl
        ---> Using cache
        ---> d91055dade53
       Step 5/16 : RUN ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key -N ''
        ---> Using cache
        ---> c647407017ec
       Step 6/16 : RUN ssh-keygen -t dsa -f /etc/ssh/ssh_host_dsa_key -N ''
        ---> Using cache
        ---> dde4401a6122
       Step 7/16 : RUN if ! getent passwd kitchen; then                 useradd -d /home/kitchen -m -s /bin/bash -p '*' kitchen;               fi
        ---> Using cache
        ---> 43bc21da1946
       Step 8/16 : RUN echo "kitchen ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
        ---> Using cache
        ---> c13e2988ada5
       Step 9/16 : RUN echo "Defaults !requiretty" >> /etc/sudoers
        ---> Using cache
        ---> 4fabb287c4c5
       Step 10/16 : RUN mkdir -p /home/kitchen/.ssh
        ---> Using cache
        ---> fa81cf0825ed
       Step 11/16 : RUN chown -R kitchen /home/kitchen/.ssh
        ---> Using cache
        ---> 6742e97caba3
       Step 12/16 : RUN chmod 0700 /home/kitchen/.ssh
        ---> Using cache
        ---> 790ecdc0c8a1
       Step 13/16 : RUN touch /home/kitchen/.ssh/authorized_keys
        ---> Using cache
        ---> 682afe7e98ab
       Step 14/16 : RUN chown kitchen /home/kitchen/.ssh/authorized_keys
        ---> Using cache
        ---> fa4279bfb34f
       Step 15/16 : RUN chmod 0600 /home/kitchen/.ssh/authorized_keys
        ---> Using cache
        ---> 4ebe38dc87f3
       Step 16/16 : RUN echo ssh-rsa\ AAAAB3NzaC1yc2EAAAADAQABAAABAQDh0jPG\+f\+JLWB9ckvab0iAhiae/ICRQ0JbIOrtwEgGsLisWLPDGE81JxXR0UuwUKuoJ30F2Mih1YgcoE7qYAouBUSseNgDs3E5O8V1IVjSGGdXXkFJf56jU7dABGbavXH3tq0l1NnBtx6krJNO5xhlJU5QhgZlbu\+1r8/g1AkEIVbWYLjaPMciicmEQ\+j\+IUNTjuMBy4rWaE\+37MmE8QgD5XL/NDkEyhZ1u2vegKebvCEcX2IhNNUnyPkVXz2lqeCyrQX/MczeguX0rtxQbJyQ3WU1/O/u5YS\+na6Td16hJt/fGQzTZkUl\+8MYqS8xFNn\+hdlZ7Fd\+QgEvs/mTYc5B\ kitchen_docker_key >> /home/kitchen/.ssh/authorized_keys
        ---> Using cache
        ---> 4ad91eb3a6dd
       Successfully built 4ad91eb3a6dd
       313775adc1572e63c1a37276e5916f5ca7b9148ead40123f576a1177b24b3f48
       0.0.0.0:32771
       [SSH] Established
       Finished creating <default-amazon-docker> (0m1.09s).
-----> Converging <default-amazon-docker>...
       Preparing files for transfer
       Preparing dna.json
       Resolving cookbook dependencies with Berkshelf 5.6.4...
       Removing non-cookbook files before transfer
       Preparing validation.pem
       Preparing client.rb
-----> Installing Chef Omnibus (12.19.36)
       Downloading https://omnitruck.chef.io/install.sh to file /tmp/install.sh
       Trying curl...
       Download complete.
       el 6 x86_64
       Getting information for chef stable 12.19.36 for el...
       downloading https://omnitruck.chef.io/stable/chef/metadata?v=12.19.36&p=el&pv=6&m=x86_64
         to file /tmp/install.sh.31/metadata.txt
       trying curl...
       sha1	323e88bd64b166a823087d0f50a949506d0fa6b5
       sha256	89e8e6e9aebe95bb31e9514052a8926f61d82067092ca3bc976b0bd223710c81
       url	https://packages.chef.io/files/stable/chef/12.19.36/el/6/chef-12.19.36-1.el6.x86_64.rpm
       version	12.19.36
       downloaded metadata file looks valid...
       downloading https://packages.chef.io/files/stable/chef/12.19.36/el/6/chef-12.19.36-1.el6.x86_64.rpm
         to file /tmp/install.sh.31/chef-12.19.36-1.el6.x86_64.rpm
       trying curl...
       Comparing checksum with sha256sum...
       Installing chef 12.19.36
       installing with rpm...
       warning: /tmp/install.sh.31/chef-12.19.36-1.el6.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 83ef826a: NOKEY
       Preparing...                          ################################# [100%]
       Updating / installing...
          1:chef-12.19.36-1.el6              ################################# [100%]
       Thank you for installing Chef!
       Transferring files to <default-amazon-docker>
       [2017-04-24T17:12:43+00:00] INFO: Forking chef instance to converge...
       Starting Chef Client, version 12.19.36
       [2017-04-24T17:12:43+00:00] INFO: *** Chef 12.19.36 ***
       [2017-04-24T17:12:43+00:00] INFO: Platform: x86_64-linux
       [2017-04-24T17:12:43+00:00] INFO: Chef-client pid: 151
       Creating a new client identity for default-amazon-docker using the validator key.
       [2017-04-24T17:12:44+00:00] INFO: Client key /tmp/kitchen/client.pem is not present - registering
       [2017-04-24T17:12:44+00:00] INFO: HTTP Request Returned 404 Not Found: Object not found: chefzero://localhost:8889/nodes/default-amazon-docker
       [2017-04-24T17:12:44+00:00] INFO: Setting the run_list to ["recipe[example_serverspec_to_inspec::default]"] from CLI options
       [2017-04-24T17:12:44+00:00] INFO: Run List is [recipe[example_serverspec_to_inspec::default]]
       [2017-04-24T17:12:44+00:00] INFO: Run List expands to [example_serverspec_to_inspec::default]
       [2017-04-24T17:12:44+00:00] INFO: Starting Chef Run for default-amazon-docker
       [2017-04-24T17:12:44+00:00] INFO: Running start handlers
       [2017-04-24T17:12:44+00:00] INFO: Start handlers complete.
       [2017-04-24T17:12:44+00:00] INFO: HTTP Request Returned 404 Not Found: Object not found:
       resolving cookbooks for run list: ["example_serverspec_to_inspec::default"]
       [2017-04-24T17:12:44+00:00] INFO: Loading cookbooks [example_serverspec_to_inspec@1.1.0]
       Synchronizing Cookbooks:
       [2017-04-24T17:12:44+00:00] INFO: Storing updated cookbooks/example_serverspec_to_inspec/metadata.json in the cache.
       [2017-04-24T17:12:44+00:00] INFO: Storing updated cookbooks/example_serverspec_to_inspec/README.md in the cache.
       [2017-04-24T17:12:44+00:00] INFO: Storing updated cookbooks/example_serverspec_to_inspec/recipes/default.rb in the cache.
       [2017-04-24T17:12:44+00:00] INFO: Storing updated cookbooks/example_serverspec_to_inspec/templates/default/sysconfig.network.erb in the cache.
         - example_serverspec_to_inspec (1.1.0)
       Installing Cookbook Gems:
       Compiling Cookbooks...
       Converging 5 resources
       Recipe: example_serverspec_to_inspec::default
         * template[/etc/sysconfig/network] action create[2017-04-24T17:12:44+00:00] INFO: Processing template[/etc/sysconfig/network] action create (example_serverspec_to_inspec::default line 7)
       [2017-04-24T17:12:44+00:00] INFO: template[/etc/sysconfig/network] created file /etc/sysconfig/network

           - create new file /etc/sysconfig/network[2017-04-24T17:12:44+00:00] INFO: template[/etc/sysconfig/network] updated file contents /etc/sysconfig/network

           - update content in file /etc/sysconfig/network from none to 9d2f64
           --- /etc/sysconfig/network	2017-04-24 17:12:44.464755578 +0000
           +++ /etc/sysconfig/.chef-network20170424-151-1caevhq	2017-04-24 17:12:44.454767098 +0000
           @@ -1 +1,4 @@
           +NETWORKING=yes
           +HOSTNAME=localhost.localdomain
           +NOZEROCONF=yes[2017-04-24T17:12:44+00:00] INFO: template[/etc/sysconfig/network] owner changed to 0
       [2017-04-24T17:12:44+00:00] INFO: template[/etc/sysconfig/network] group changed to 0
       [2017-04-24T17:12:44+00:00] INFO: template[/etc/sysconfig/network] mode changed to 644

           - change mode from '' to '0644'
           - change owner from '' to 'root'
           - change group from '' to 'root'
         * yum_package[php70-fpm, nginx] action install[2017-04-24T17:12:44+00:00] INFO: Processing yum_package[php70-fpm, nginx] action install (example_serverspec_to_inspec::default line 15)
       [2017-04-24T17:12:49+00:00] INFO: yum_package[php70-fpm, nginx] installing php70-fpm-7.0.16-1.21.amzn1 from amzn-main repository nginx-1.10.2-1.30.amzn1 from amzn-main repository
       [2017-04-24T17:12:53+00:00] INFO: yum_package[php70-fpm, nginx] installed ["php70-fpm", "nginx"] at ["7.0.16-1.21.amzn1", "1.10.2-1.30.amzn1"]

           - install version 7.0.16-1.21.amzn1 of package php70-fpm
           - install version 1.10.2-1.30.amzn1 of package nginx
         * ruby_block[Set PHP FPM Ownership] action run[2017-04-24T17:12:53+00:00] INFO: Processing ruby_block[Set PHP FPM Ownership] action run (example_serverspec_to_inspec::default line 17)
       [2017-04-24T17:12:53+00:00] WARN: Open3: Standard Out ()
       [2017-04-24T17:12:53+00:00] WARN: Open3: Status (pid 429 exit 0)
       [2017-04-24T17:12:53+00:00] INFO: ruby_block[Set PHP FPM Ownership] called

           - execute the ruby block Set PHP FPM Ownership
         * service[php-fpm-7.0] action enable[2017-04-24T17:12:53+00:00] INFO: Processing service[php-fpm-7.0] action enable (example_serverspec_to_inspec::default line 32)
       [2017-04-24T17:12:53+00:00] INFO: service[php-fpm-7.0] enabled

           - enable service service[php-fpm-7.0]
         * service[php-fpm-7.0] action start[2017-04-24T17:12:53+00:00] INFO: Processing service[php-fpm-7.0] action start (example_serverspec_to_inspec::default line 32)
       [2017-04-24T17:12:53+00:00] INFO: service[php-fpm-7.0] started

           - start service service[php-fpm-7.0]
         * service[nginx] action enable[2017-04-24T17:12:53+00:00] INFO: Processing service[nginx] action enable (example_serverspec_to_inspec::default line 37)
       [2017-04-24T17:12:53+00:00] INFO: service[nginx] enabled

           - enable service service[nginx]
         * service[nginx] action start[2017-04-24T17:12:53+00:00] INFO: Processing service[nginx] action start (example_serverspec_to_inspec::default line 37)
       [2017-04-24T17:12:53+00:00] INFO: service[nginx] started

           - start service service[nginx]
       [2017-04-24T17:12:53+00:00] INFO: Chef Run complete in 9.47905186 seconds

       Running handlers:
       [2017-04-24T17:12:53+00:00] INFO: Running report handlers
       Running handlers complete
       [2017-04-24T17:12:53+00:00] INFO: Report handlers complete
       Chef Client finished, 7/7 resources updated in 10 seconds
       Finished converging <default-amazon-docker> (0m28.03s).
-----> Setting up <default-amazon-docker>...
       Finished setting up <default-amazon-docker> (0m0.00s).
-----> Verifying <default-amazon-docker>...
       Detected alternative framework tests for `inspec`
       Loaded tests from {:path=>"/Users/levon/Development/github/bonusbits/example_serverspec_to_inspec/test/integration/default/inspec"}

Profile: tests from {:path=>"/Users/levon/Development/github/bonusbits/example_serverspec_to_inspec/test/integration/default/inspec"}
Version: (not specified)
Target:  ssh://kitchen@localhost:32771


  Nginx nginx
       installed
       service
  Php FPM
       php-fpm installed
       php-fpm service
       nginx owns /var/log/php-fpm
       nginx owns /var/lib/php/7.0

Test Summary: 6 successful, 0 failures, 0 skipped
       Finished verifying <default-amazon-docker> (0m0.91s).
-----> Destroying <default-amazon-docker>...
       PID                 USER                TIME                COMMAND
       4263                root                0:00                /usr/sbin/sshd -D -o UseDNS=no -o UsePAM=no -o PasswordAuthentication=yes -o UsePrivilegeSeparation=no -o PidFile=/tmp/sshd.pid
       4293                root                0:00                sshd: kitchen@pts/0
       4779                root                0:00                {php-fpm-7.0} php-fpm: master process (/etc/php-fpm.conf)
       4780                499                 0:00                {php-fpm-7.0} php-fpm: pool www
       4781                499                 0:00                {php-fpm-7.0} php-fpm: pool www
       4782                499                 0:00                {php-fpm-7.0} php-fpm: pool www
       4783                499                 0:00                {php-fpm-7.0} php-fpm: pool www
       4784                499                 0:00                {php-fpm-7.0} php-fpm: pool www
       4855                root                0:00                nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
       4856                499                 0:00                nginx: worker process
       4857                499                 0:00                nginx: worker process
       4858                499                 0:00                nginx: worker process
       4860                499                 0:00                nginx: worker process
       4861                root                0:00                sshd: kitchen@notty
       313775adc1572e63c1a37276e5916f5ca7b9148ead40123f576a1177b24b3f48
       313775adc1572e63c1a37276e5916f5ca7b9148ead40123f576a1177b24b3f48
       Finished destroying <default-amazon-docker> (0m1.20s).
       Finished testing <default-amazon-docker> (0m31.26s).
-----> Kitchen is finished. (0m32.26s)

Gnome-sticky-notes-applet Notice that it took 32 seconds to do the test. Whereas, the same tests ran as ServerSpec took 1.5 minutes.


Reference Github Link


Walkthrough Video


Related Articles


Sources