Configure Test Kitchen to Use ChefDK Chef Client

From Bonus Bits
Jump to: navigation, search

Purpose

This article gives an example of how to use ChefDK Chef Client with Test Kitchen AWS EC2 Driver. Under the yaml_anchors section, there are a few examples not all called by this config to show various methods to fetch chefdk and install in the user data section.


Environment

  • ChefDK 2.4.17, 2.5.13 (Not 2.5.3 because there is a YAML Anchor parsing bug)
  • MacOS 10.13.5


Prerequisites


Kitchen Config (New Provisioner Settings)

Without spending a lot of time looking through changelogs the bottom line is at some newer version including ChefDK 2.5.13 there are new provisioner settings to help ease the pain of using the User Data to run your own home brewed chefdk install. This is great because it solves the problem of having to wait for the user data script to finish before running converge+!

Summary

  1. Use new provisioner settings to set as chefdk and optionally point to an internal download URL
  2. Set Chef-Client Path to the ChefDK path
    provisioner
      product_name: chefdk
      product_version: 2.5.13
      download_url: http://artifactory.domain.com/artifactory/chef-deb/stable/ubuntu/16.04/chefdk_2.5.13-1_amd64.deb
    
---

driver:
  name: ec2
  aws_ssh_key_id: <%= ENV.fetch('AWS_SSH_KEY_ID', 'bonusbits_dev_key') %>
  iam_profile_name: <%= ENV.fetch('AWS_IAM_INSTANCE_PROFILE_NIX', 'Linux-Instance_Role') %>
  instance_initiated_shutdown_behavior: terminate
  instance_type: t2.micro
  region: <%= ENV.fetch('AWS_REGION', 'us-west-2') %>
  security_group_ids: 
    - <%= ENV.fetch('AWS_SECURITY_GROUP_NIX_1', 'sg-00000000') %>
    - <%= ENV.fetch('AWS_SECURITY_GROUP_NIX_2', 'sg-00000000') %>
  subnet_id: <%= ENV.fetch('AWS_SUBNET_ID', 'subnet-00000000') %>
  vpc_id: <%= ENV.fetch('AWS_VPC_ID', 'vpc-00000000') %>    

platforms:
  - name: amazon-ami
    driver:
      image_search:
        owner-id: 137112412989
        name: amzn-ami-hvm-*x86_64-gp2

transport:
  username: ec2-user
  ssh_key: <%= ENV.fetch('AWS_SSH_KEY_PATH', ENV['HOME'] + "/.ssh/" + ENV['AWS_SSH_KEY_ID'] + ".pem") %>

provisioner:
  name: chef_zero

suites:
  - name: amazon
    driver:
      tags:
        Name: kitchen-amazon-<%= ENV['USER'] %>
        Created-By: Test Kitchen
        Owner: <%= ENV.fetch('USER', 'bonusbits') %>
    provisioner
      product_name: chefdk
      product_version: 2.5.13
      download_url: http://artifactory.domain.com/artifactory/chef-deb/stable/ubuntu/16.04/chefdk_2.5.13-1_amd64.deb
    run_list:
      - recipe[mycookbook]

verifier:
  name: inspec

Kitchen Config (User Data)

This configuration contains three OS types. Amazon Linux latest, Ubuntu 16.04 and Windows 2016. There are two test suites. One is for the EC2 driver while the other is for the Docker driver. Example Inspec testing is included. Along with the example of the folder structure for the local Inspec profile. Notice the bootstrap yaml_anchors. They install ChefDK for Linux or Windows. Then on the platforms, the ChefDK Chef-Client path is specified so Kitchen knows the path to call.

Gnome-sticky-notes-applet Since we are installing ChefDK with shell scripting in the User Data section, we can't immediately run Chef Client (kitchen converge) on the instance. This is because the instance needs time to finish running the User Data Script after it's initialized. So, as a workaround simply do kitchen create Wait 5 or so minutes or login and watch /var/log/cloud-init-output.log to finish. Then run kitchen converge or kitchen verify. If you do this from a CI pipeline you could split the kitchen test command out into it's steps and add a sleep between create and converge.

Summary

  1. Pre-install ChefDK with User Data
  2. Set Chef-Client Path to the ChefDK path
    provisioner:
      chef_client_path: "/opt/chefdk/bin/chef-client"
    
---

# Environment Configurations - YAML Anchors
yaml_anchors:
  # AWS Account
  aws_account_id: &aws_account_id <%= ENV.fetch('AWS_ACCOUNT_ID', '000000000000') %>
  aws_iam_instance_profile_nix: &aws_iam_instance_profile_nix <%= ENV.fetch('AWS_IAM_INSTANCE_PROFILE_NIX', 'Linux-Instance_Role') %>
  aws_iam_instance_profile_win: &aws_iam_instance_profile_win <%= ENV.fetch('AWS_IAM_INSTANCE_PROFILE_WIN', 'Windows-Instance_Role') %>
  aws_public_ip: &aws_public_ip <%= ENV.fetch('AWS_PUBLIC_IP', 'false') %>
  aws_region: &aws_region <%= ENV.fetch('AWS_REGION', 'us-west-2') %>
  aws_security_groups_nix: &aws_security_groups_nix
    - <%= ENV.fetch('AWS_SECURITY_GROUP_NIX_1', 'sg-00000000') %>
    - <%= ENV.fetch('AWS_SECURITY_GROUP_NIX_2', 'sg-00000000') %>
  aws_security_groups_win: &aws_security_groups_win
    - <%= ENV.fetch('AWS_SECURITY_GROUP_WIN_1', 'sg-00000000') %>
    - <%= ENV.fetch('AWS_SECURITY_GROUP_WIN_2', 'sg-00000000') %>
  aws_ssh_key_id: &aws_ssh_key_id <%= ENV.fetch('AWS_SSH_KEY_ID', 'bonusbits_dev_key') %>
  aws_ssh_key_path: &aws_ssh_key_path <%= ENV.fetch('AWS_SSH_KEY_PATH', ENV['HOME'] + "/.ssh/" + ENV['AWS_SSH_KEY_ID'] + ".pem") %>
  aws_subnet_id: &aws_subnet_id <%= ENV.fetch('AWS_SUBNET_ID', 'subnet-00000000') %>
  aws_tag_owner: &aws_tag_owner <%= ENV.fetch('USER', 'bonusbits') %>
  aws_vpc_id: &aws_vpc_id <%= ENV.fetch('AWS_VPC_ID', 'vpc-00000000') %>

  # Bootstrap ChefDK
  bootstrap_noproxy: &bootstrap_noproxy
    #!/usr/bin/env bash

    # Install ChefDK from Internet
    ChefDKVersion=2.4.17
    curl -L https://omnitruck.chef.io/install.sh | bash -s -- -P chefdk -v ${ChefDkVersion}
  bootstrap_artifactory_deb: &bootstrap_artifactory_deb
    #!/usr/bin/env bash

    # Install ChefDK from Artifactory
    CHEF_BIN=chefdk_2.5.13-1_amd64.deb
    ARTIFACT_URL=https://artifactory.domain.com/artifactory/chef/ubuntu
    wget ${ARTIFACT_URL}/${CHEF_BIN} -P /tmp/ && dpkg -i /tmp/${CHEF_BIN}
  bootstrap_s3_rpm: &bootstrap_s3_rpm
    #!/usr/bin/env bash

    # Install ChefDK from S3
    CHEFDK_BIN=chefdk-2.5.13-1.el7.x86_64.rpm
    ARTIFACT_BUCKET=my-bucket
    aws s3 cp s3://${ARTIFACT_BUCKET}/chef/bin/${CHEFDK_BIN} /tmp/ && rpm -Uvh /tmp/${CHEFDK_BIN}
  bootstrap_win: &bootstrap_win
    <powershell>
      $DeployBucket = 'bonubits_deploy'
      $ChefDkMsi = 'chefdk-2.4.17-1-x86.msi'
      net stop winrm
      net user testkitchen "P@ssword!" /add
      net localgroup "Administrators" /add "bonusbits\<%= ENV['USER'] %>" testkitchen
      net localgroup "Remote Desktop Users" /add "bonusbits\<%= ENV['USER'] %>" testkitchen
      Copy-S3Object -BucketName $DeployBucket -Key "binaries/$ChefDkMsi" -LocalFile "C:\chef\cache\$ChefDkMsi"
      Start-Process -FilePath "C:\chef\cache\$ChefDkMsi" -ArgumentList "/qn" -Wait
      Wait-Process -Name 'msiexec' -Timeout 380
      $OrgPath = (Get-ItemProperty -Path 'Registry::HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment' -Name PATH).path
      $NewPath = "C:\opscode\chefdk\bin;C:\opscode\chefdk\embedded;C:\opscode\chefdk\embedded\bin;$OrgPath"
      Set-ItemProperty -Path 'Registry::HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment' -Name PATH -Value $NewPath
      net start winrm
    </powershell>

platforms:
  # AWS AMI/EC2 Platforms
  - name: amazon-ami
    driver:
      aws_ssh_key_id: *aws_ssh_key_id
      iam_profile_name: *aws_iam_instance_profile_nix
      image_search:
        owner-id: 137112412989
        name: amzn-ami-hvm-*x86_64-gp2
      instance_initiated_shutdown_behavior: terminate
      instance_type: t2.micro
      name: ec2
      region: *aws_region
      security_group_ids: *aws_security_groups_nix
      subnet_id: *aws_subnet_id
      tags:
        Created-By: Test Kitchen
        OS: Amazon 2017
        Owner: *aws_tag_owner
      user_data: *bootstrap_noproxy
      vpc_id: *aws_vpc_id
    provisioner:
      chef_client_path: "/opt/chefdk/bin/chef-client"
    transport:
      username: ec2-user
      ssh_key: *aws_ssh_key_path

  - name: ubuntu-16.04-ami
    driver:
      associate_public_ip: *aws_public_ip
      aws_ssh_key_id: *aws_ssh_key_id
      # http_proxy: *http_proxy
      # https_proxy: *https_proxy
      iam_profile_name: *aws_iam_instance_profile_nix
      image_search:
        owner-id: 099720109477
        name: ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server*
      instance_initiated_shutdown_behavior: terminate
      instance_type: t2.micro
      name: ec2
      region: *aws_region
      security_group_ids: *aws_security_groups_nix
      subnet_id: *aws_subnet_id
      tags:
        Created-By: Test Kitchen
        OS: Ubuntu 16.04
        Owner: *aws_tag_owner
      user_data: *bootstrap_artifactory_deb
      vpc_id: *aws_vpc_id
    provisioner:
      chef_client_path: "/opt/chefdk/bin/chef-client"
    transport:
      username: ubuntu
      ssh_key: *aws_ssh_key_path

  - name: windows-2016-ami
    driver:
      associate_public_ip: *aws_public_ip
      aws_ssh_key_id: *aws_ssh_key_id
      iam_profile_name: *aws_iam_instance_profile_win
      image_search:
        owner-id: 801119661308
        name: Windows_Server-2016-English-Full-Base*
      instance_initiated_shutdown_behavior: terminate
      instance_type: m5.xlarge
      name: ec2
      region: *aws_region
      security_group_ids: *aws_security_groups_win
      subnet_id: *aws_subnet_id
      tags:
        Created-By: Test Kitchen
        OS: Windows 2016
        Owner: *aws_tag_owner
      user_data: *bootstrap_win
      vpc_id: *aws_vpc_id
    provisioner:
      chef_client_path: "C:\\opscode\\chefdk\\bin\\chef-client.bat"
    transport:
      password: "P@ssword!"
      ssh_key: *aws_ssh_key_path
      username: testkitchen

  # Docker Platforms (kitchen-docker gem required)
  - name: amazon-container
    driver:
      name: docker
      use_sudo: false # For Native Docker on Mac. Remove/Comment if using Toolbox (docker-machine)
      image: amazonlinux:latest
      platform: rhel
      provision_command:
        - yum -y install upstart procps util-linux
        - *bootstrap_noproxy
      ssl_verify_mode: ":verify_none"
    provisioner:
      chef_client_path: "/opt/chefdk/bin/chef-client"

  - name: ubuntu-16.04-container
    driver:
      name: docker
      use_sudo: false # For Native Docker on Mac. Remove/Comment if using Toolbox (docker-machine)
      image: ubuntu:16.04
      platform: ubuntu
      provision_command:
        - apt update && apt install upstart procps util-linux
        - *bootstrap_artifactory_deb
      ssl_verify_mode: ":verify_none"
    provisioner:
      chef_client_path: "/opt/chefdk/bin/chef-client"

  - name: windows-2016-container
    driver:
      name: docker
      use_sudo: false # For Native Docker on Mac. Remove/Comment if using Toolbox (docker-machine)
      image: microsoft/windowsservercore:ltsc2016
      platform: windows
      provision_command:
        - *bootstrap_win
      ssl_verify_mode: ":verify_none"
    provisioner:
      chef_client_path: "C:\\opscode\\chefdk\\bin\\chef-client.bat"

provisioner:
  always_update_cookbooks: true
  data_bags_path: "test/data_bags"
  encrypted_data_bag_secret_key_path: "test/data_bags/encrypted_data_bag_secret"
  environments_path: "test/environments"
  name: chef_zero
  roles_path: "test/roles"

suites:
  - name: ec2
    driver:
      tags:
        Name: kitchen-bonusbits-base-<%= ENV['USER'] %>
        Created-By: Test Kitchen
    includes: ["amazon-ami", "ubuntu-16.04-ami", "windows-2016-ami"]
    provisioner:
      client_rb:
        environment: bonusbits_base
    run_list:
      - role[base]
    verifier:
      attributes:
        configure_backups: 'true'

  - name: docker
    attributes:
      <% if ENV['CIRCLECI'] %>
      bonusbits_base:
        deployment_location: 'circleci'
      <% end %>
    run_list:
      - role[base]
    provisioner:
      client_rb:
        environment: bonusbits_base
    includes: ["amazon-container", "ubuntu-16.04-container", "windows-2016-container"]

verifier:
  name: inspec
  format: <%= ENV['CI'] ? 'junit' : 'cli' %>
  <% if ENV['CI'] %>
  output: "reports/%{platform}_%{suite}_inspec.xml"
  <% end %>
  inspec_tests:
    - path: test/inspec/base
  attributes:
    debug: 'false'
    chef_version: '13.6.4'

Test Folder Structure Example

.
├── test
   └── inspec
       └── base
               ├── CHANGELOG.md
               ├── controls
               │   └── aws.rb
               │   ├── cloudwatch_logs.rb
               │   ├── epel.rb
               │   ├── node_info.rb
               │   ├── packages.rb
               │   ├── proxy.rb
               │   ├── selinux.rb
               │   ├── sudoers.rb
               │   └── yum_cron.rb
               ├── helpers
               │   └── node_attributes.rb
               │   └── os_queries.rb
               └── inspec.yml

Kitchen List Example

kitchen list

Instance                       Driver  Provisioner  Verifier  Transport  Last Action    Last Error
ec2-amazon-ami                 Ec2     ChefZero     Inspec    Ssh        <Not Created>  <None>
ec2-ubuntu-1604-ami            Ec2     ChefZero     Inspec    Ssh        <Not Created>  <None>
ec2-windows-2016-ami           Ec2     ChefZero     Inspec    Winrm      <Not Created>  <None>
docker-amazon-container        Docker  ChefZero     Inspec    Ssh        <Not Created>  <None>
docker-ubuntu-1604-container   Docker  ChefZero     Inspec    Ssh        <Not Created>  <None>
docker-windows-2016-container  Docker  ChefZero     Inspec    Winrm      <Not Created>  <None>


Kitchen Test Examples

Test All

kitchen test

Test All EC2

kitchen test ec2

Test All Docker

kitchen test docker

Test Single

kitchen test docker-amazon-container


Related Articles


Sources