Setup Docker Container to Use Charles Proxy on macOS

From Bonus Bits
Jump to: navigation, search

Purpose

This article gives a way to setup a local Docker container to use local Charles Proxy as it's proxy network on macOS. Charles Proxy containers the auth and enterprise proxy info and forwards the traffic sent to it on default TCP port 8888. This is for all you lucky individuals stuck behind a network proxy. The purpose is to allow internet and internal network access from our containers. Such as installing packages with yum/apt-get, pulling binaries from S3/Artifactory or Github access.


Prerequisites


Create Unique Local IP on Loopback Interface on the Host (macOS)

The standard loopback IP or 'localhost' obviously won't work from the container because it would just look at it self. We could use our host IP, but then it's static to our system and if the IP changes it would fail. So we are going to assign an IP to the loopback that will be consistent.

  1. Open a terminal
  2. Add an IP alias to Loopback Interface
    sudo ifconfig lo0 inet alias 169.254.169.254/32
    

Bash Functions to Enable or Disable Loopback Alias

Here's a simple set of Bash functions you can add to your profile to enable or disable the loopback alias.

# Create Loopback Alias
function proxy-enable-lo-alias(){
  echo "Enabling Loopback Aliases..."
  if ! ifconfig -L lo0 | grep 169.254.169.254
  then
    sudo ifconfig lo0 inet alias 169.254.169.254/32
    echo "Loopback Alias Created!"
    echo "Now Reconnect VPN and Restart Charles Proxy"
  else
    echo "Loopback Alias Already Created"
  fi
}

# Remove Loopback Alias
function proxy-disable-lo-alias(){
  echo "Disabling Loopback Aliases..."
  if ifconfig -L lo0 | grep 169.254.169.254
  then
    sudo ifconfig lo0 inet -alias 169.254.169.254
    echo "Loopback Alias Removed!"
    echo "Now Reconnect VPN and Restart Charles Proxy"
  else
    echo "Loopback Alias Already Removed"
  fi
}


Set Host CLI to Use Alias (Optional)

Because Test Kitchen pulls the CLI proxy environment variables and likely we'd be setting configurations to use our environment variables for proxy so it's more dynamic. I find it's best practice to switch you CLI proxy environment variables to use the new IP Alias.

  1. Change Bash Profile to use 169.254.169.254 instead of localhost

Example

Add environment variables to ~/.bash_profile

# Show Current Settings
function proxy-show(){
  echo "ALL_PROXY = ($ALL_PROXY)"
  echo "HTTP_PROXY = ($HTTP_PROXY)"
  echo "http_proxy = ($http_proxy)"
  echo "HTTPS_PROXY = ($HTTPS_PROXY)"
  echo "https_proxy = ($https_proxy)"
  echo "ftp_proxy = ($ftp_proxy)"
  echo "rsync_proxy = ($rsync_proxy)"
  echo "no_proxy = ($no_proxy)"
  echo "NO_PROXY = ($NO_PROXY)"
}

# Enable Proxy
function proxy-on(){
  if ifconfig -L lo0 | grep 169.254.169.254
  then
    export ALL_PROXY=http://169.254.169.254:8888
  else
    export ALL_PROXY=http://localhost:8888
  fi
  export HTTP_PROXY=$ALL_PROXY
  export http_proxy=$ALL_PROXY
  export HTTPS_PROXY=$ALL_PROXY
  export https_proxy=$ALL_PROXY
  export ftp_proxy=$ALL_PROXY
  export rsync_proxy=$ALL_PROXY
  export NO_PROXY=localhost,.local,169.254.,127.0.0.1,10.0.2.,/var/run/docker.sock,.bonusbits.com
  export no_proxy=$NO_PROXY
  proxy-show
}

# Disable Proxy
function proxy-off(){
  unset HTTP_PROXY HTTPS_PROXY http_proxy https_proxy ftp_proxy rsync_proxy ALL_PROXY NO_PROXY no_proxy
  proxy-show
}

# Turn on by default
proxy-enable-lo-alias
proxy-on


Configure Docker Image/Container to Use Proxy

Manually

First, we would most likely manually configure the proxy on a container to test our setup works.

  1. Launch Container Interactive Shell
    docker run -it <imagename>:<tag> /bin/bash
    
  2. Set Proxy
    export http_proxy="http://169.254.169.254:8888"
    
    export https_proxy="http://169.254.169.254:8888"
    
    export no_proxy="localhost,.local,127.0.0.1,169.254.,10.0.2.,/var/run/docker.sock,.bonusbits.com"
    


Dockerfile

ENV ALL_PROXY=http://169.254.169.254:8888
ENV HTTP_PROXY=$ALL_PROXY
ENV HTTPS_PROXY=$ALL_PROXY
ENV http_proxy=$ALL_PROXY
ENV https_proxy=$ALL_PROXY
ENV NO_PROXY=localhost,.local,169.254.,169.254.169.254,127.0.0.1,10.0.2.,192.168.25.,192.168.99.,.kdc.capitalone.com,.ds.capitalone.com,.cloud.capitalone.com,/var/run/docker.sock
ENV no_proxy=$NO_PROXY


Test Kitchen

(Not Fully Tested)

Example where EC2 and Docker configured in same Kitchen configuration file.

anchors:
  # AWS Account
  aws_account_id: &aws_account_id <%= ENV.fetch('AWS_ACCOUNT_ID', '000000000000') %>
  aws_iam_instance_prifile_nix: &aws_iam_instance_prifile_nix <%= ENV.fetch('AWS_IAM_INSTANCE_PROFILE_NIX', 'Linux-Instance_Role') %>
  aws_iam_instance_prifile_win: &aws_iam_instance_prifile_win <%= ENV.fetch('AWS_IAM_INSTANCE_PROFILE_WIN', 'Windows-Instance_Role') %>
  aws_public_ip: &aws_public_ip <%= ENV.fetch('AWS_PUBLIC_IP', 'false') %>
  aws_region: &aws_region <%= ENV.fetch('AWS_REGION', 'us-west-2') %>
  aws_security_groups_nix: &aws_security_groups_nix
    - <%= ENV.fetch('AWS_SECURITY_GROUP_NIX_1', 'sg-00000000') %>
    - <%= ENV.fetch('AWS_SECURITY_GROUP_NIX_2', 'sg-00000000') %>
  aws_security_groups_win: &aws_security_groups_win
    - <%= ENV.fetch('AWS_SECURITY_GROUP_WIN_1', 'sg-00000000') %>
    - <%= ENV.fetch('AWS_SECURITY_GROUP_WIN_2', 'sg-00000000') %>
  aws_ssh_key_id: &aws_ssh_key_id <%= ENV.fetch('AWS_SSH_KEY_ID', 'bonusbits_dev_key') %>
  aws_ssh_key_path: &aws_ssh_key_path <%= ENV.fetch('AWS_SSH_KEY_PATH', ENV['HOME'] + "/.ssh/" + ENV['AWS_SSH_KEY_ID'] + ".pem") %>
  aws_subnet_id: &aws_subnet_id <%= ENV.fetch('AWS_SUBNET_ID', 'subnet-00000000') %>
  aws_tag_owner: &aws_tag_owner <%= ENV.fetch('USER', 'bonusbits') %>
  aws_vpc_id: &aws_vpc_id <%= ENV.fetch('AWS_VPC_ID', 'vpc-00000000') %>

  # Bootstrap ChefDK
  bootstrap_nix: &aws_userdata_nix
    #!/usr/bin/env bash
    ChefDKVersion=2.4.17
    export http_proxy=*aws_http_proxy
    export https_proxy=*aws_https_proxy
    export no_proxy=*aws_no_proxy
    curl -L https://omnitruck.chef.io/install.sh | bash -s -- -P chefdk -v ${ChefDkVersion}
  bootstrap_win: &aws_userdata_win
    <powershell>
      $DeployBucket = 'bonubits_deploy'
      $ChefDkMsi = 'chefdk-2.4.17-1-x86.msi'
      $env:http_proxy = "*aws_http_proxy"
      $env:https_proxy = "*aws_https_proxy"
      $env:no_proxy = "*aws_no_proxy"
      net stop winrm
      net user testkitchen "P@ssword!" /add
      net localgroup "Administrators" /add "bonusbits\<%= ENV['USER'] %>" testkitchen
      net localgroup "Remote Desktop Users" /add "bonusbits\<%= ENV['USER'] %>" testkitchen
      Copy-S3Object -BucketName $DeployBucket -Key "binaries/$ChefDkMsi" -LocalFile "C:\chef\cache\$ChefDkMsi"
      Start-Process -FilePath "C:\chef\cache\$ChefDkMsi" -ArgumentList "/qn" -Wait
      Wait-Process -Name 'msiexec' -Timeout 380
      $OrgPath = (Get-ItemProperty -Path 'Registry::HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment' -Name PATH).path
      $NewPath = "C:\opscode\chefdk\bin;C:\opscode\chefdk\embedded;C:\opscode\chefdk\embedded\bin;$OrgPath"
      Set-ItemProperty -Path 'Registry::HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment' -Name PATH -Value $NewPath
      net start winrm
    </powershell>

  # Proxy
  local_http_proxy: &local_http_proxy <%= ENV.fetch('HTTP_PROXY', 'http://169.254.169.254:8888') %>
  local_https_proxy: &local_https_proxy <%= ENV.fetch('HTTPS_PROXY', 'http://169.254.169.254:8888') %>
  local_no_proxy: &local_no_proxy <%= ENV.fetch('NO_PROXY', 'localhost,127.0.0.1,169.254.169.254,10.0.2.,/var/run/docker.sock,.bonusbits.com') %>
  aws_http_proxy: &aws_http_proxy <%= ENV.fetch('HTTP_PROXY', 'http://proxy.bonusbits.com:8088') %>
  aws_https_proxy: &aws_https_proxy <%= ENV.fetch('HTTPS_PROXY', 'https://proxy.bonusbits.com:8099') %>
  aws_no_proxy: &aws_no_proxy <%= ENV.fetch('NO_PROXY', 'localhost,127.0.0.1,169.254.169.254,10.0.2.,.s3.amazonaws.com,/var/run/docker.sock,.bonusbits.com') %>

verifier:
  name: inspec
  format: <%= ENV['CI'] ? 'junit' : 'cli' %>
  <% if ENV['CI'] %>
  output: "reports/%{platform}_%{suite}_inspec.xml"
  <% end %>
  inspec_tests:
    - name: bootstrap
      git: https://github.com/bonusbits/inspec_bootstrap.git
    - name: bonusbits_base
      git: https://github.com/bonusbits/inspec_bonusbits_base.git
    # - path: ../inspec_bonusbits_base
  attributes:
    debug: 'false'
    chef_version: '13.6.4'

provisioner:
  # always_update_cookbooks: true
  chef_client_path: "/opt/chefdk/bin/chef-client"
  data_bags_path: "test/data_bags"
  encrypted_data_bag_secret_key_path: "test/data_bags/encrypted_data_bag_secret"
  environments_path: "test/environments"
  name: chef_zero
  roles_path: "test/roles"

platforms:
  # AWS AMI/EC2 Platforms
  - name: amazon-ami
    driver:
      associate_public_ip: *aws_public_ip
      aws_ssh_key_id: *aws_ssh_key_id
      http_proxy: *aws_http_proxy
      https_proxy: *aws_https_proxy
      iam_profile_name: *aws_iam_instance_prifile_nix
      image_search:
        owner-id: 137112412989
        name: amzn-ami-hvm-2017.*x86_64-gp2
      instance_initiated_shutdown_behavior: terminate
      instance_type: t2.micro
      name: ec2
      no_proxy: *aws_no_proxy
      region: *aws_region
      security_group_ids: *aws_security_groups_nix
      subnet_id: *aws_subnet_id
      tags:
        Created-By: Test Kitchen
        OS: Amazon 2017
        Owner: *aws_tag_owner
      user_data: *bootstrap_nix
      vpc_id: *aws_vpc_id
    transport:
      username: ec2-user
      ssh_key: *aws_ssh_key_path

  # Docker Platforms
  - name: amazon-docker
    driver:
      http_proxy: *local_http_proxy
      https_proxy: *local_https_proxy
      image: amazonlinux:latest
      name: docker
      no_proxy: *local_no_proxy
      platform: rhel
      provision_command:
        - yum -y install upstart procps util-linux
      ssl_verify_mode: ":verify_none"
      use_sudo: false # For Native Docker on Mac. Remove/Comment if using Toolbox (docker-machine)

suites:
  - name: ec2_base
    provisioner:
      client_rb:
        environment: bonusbits_base
    driver:
      tags:
        Name: kitchen-bonusbits-base-<%= ENV['USER'] %>
    run_list:
      - role[base]
    verifier:
      attributes:
        configure_backups: 'true'
    includes: ["amazon-ami"]

  - name: docker_base
    run_list:
      - role[base]
    provisioner:
      client_rb:
        environment: bonusbits_base
    attributes:
      <% if ENV['CIRCLECI'] %>
      bonusbits_base:
        deployment_location: 'circleci'
      <% end %>
    includes: ["amazon-docker"]