Category: configuration management

kitchen-gce 0.1.0

I’ve just released version 0.1.0 of kitchen-gce to RubyGems.org. Kitchen-gce is a Google Compute Engine driver for Test-Kitchen; big changes in this release are:

As always, feedback and pull requests are welcome!

Kitchen-gce 0.0.6 Released

Just a quick post to announce: I’ve released kitchen-gce 0.0.6, the latest version of my Google Compute Engine driver for Test Kitchen. This release adds support for specifying the test instance’s network in GCE, and for tagging instances.

Kitchen-gce currently has a dependency on Fog 1.19.0 (only); the next release will support Fog 1.20.0 (and greater), but will likely require changes to .kitchen.yml. More details in the GitHub issue.

Introducing kitchen-gce – a Test Kitchen Driver for Google Compute Engine

Test Kitchen is an integration test framework for, among other things, Chef development. To create its testing environment, it builds on virtualization tools and cloud providers such as Docker, EC2, Rackspace and Vagrant. My own contribution – currently in 0.0.1 release, but undergoing active development – is kitchen-gce, a driver for Google Compute Engine (GCE).

Compared to other cloud providers, for use with Test Kitchen GCE has a couple of advantages, namely:

  • (Subjectively) faster instance launch times than other providers; and
  • Sub-hour billing.

One brief usage note:

Kitchen-gce uses Fog to interface with GCE. Fog’s GCE provider code has been undergoing development, and until then next releases of Fog and kitchen-gce come out (greater than 1.18.0 and 0.0.1, respectively), I recommend using the development versions of each. For instance, in a Gemfile:

group :integration do
gem 'test-kitchen'
gem 'kitchen-gce', :git => 'https://github.com/anl/kitchen-gce.git'
gem 'fog', :git => 'https://github.com/fog/fog.git', :ref => 'c5e6e2ae868b7fdec8e4fd8ef729fcab3b199b15'
end

If you use Test Kitchen, please check out kitchen-gce. Feedback, issues and pull requests welcome.

Acknowledgements: Google has currently given me a free trial plan for GCE, and has, as of this writing, covered about $0.18 of compute time on GCE used in developing kitchen-gce. I spent about $0.55 of my own money before that – sub-hour billing really does make economic sense…

Puppet Module Development with Vagrant

By way of background, I recently switched employers, and in that change, made the switch from Puppet to Chef at work. In doing so, I’ve become a big fan of a Vagrant-driven development workflow. However, on my own time, I still hack around with Puppet – and find myself wanting to use a similar development process when writing modules as opposed to cookbooks.

As far as my searching could could turn up, most of the documentation out on the web pertaining to using Puppet and Vagrant together is about using the Vagrant Puppet provisioner to set up an application development environment, not for using Vagrant to actively test Puppet modules during their development. This was momentarily frustrating to me, but the configuration I came up with to do this is pretty simple. My solution? Use a slightly non-standard module directory layout, add an additional “vagrant.pp” file, and include this in your Vagrantfile:

  config.vm.provision :puppet do |puppet|
    puppet.manifests_path = 'vagrant'
    puppet.manifest_file = 'vagrant.pp'
    puppet.module_path = '../'
  end

The Puppet module directory layout looks like this:

module/
├── Gemfile
├── Gemfile.lock
├── Modulefile
├── README
├── Vagrantfile
├── manifests
│   └── init.pp
├── spec
│   └── spec_helper.rb
├── tests
│   └── init.pp
└── vagrant
    └── vagrant.pp

In particular, in addition to the Vagrantfile, note the “vagrant” directory and the “vagrant.pp” file contained within it; said “vagrant.pp” file is short and sweet:

# Include this module
include <module>

(Where “<module>” is replaced with the actual module name.)

Finally, note that your development directory name must match the class name – otherwise the Puppet provisioner will be unable to find the included class. So, if you namespace your Puppet modules on GitHub with a leading “puppet-” or something similar, you’ll need to rename your development directory.

This layout does conflict with the Puppet module layout specification (by adding additional files/directories), although I haven’t yet noticed any ill effects. I expect – and hope – that test-kitchen for Puppet will render this blog post obsolete in the near future.

Edit 9/28/2013, 10/1/2013: If you are running a somewhat dated Vagrant box, you may find that “package” resources fail, ultimately with an error similar to the following:

E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

If updating your box isn’t an immediate option, you can work around this on Ubuntu by adding the following to your Vagrantfile before the Puppet provisioner block:

config.vm.provision :shell, :inline => '/usr/bin/apt-get update'

To avoid running this every time, you can wrap it in a test for an environment variable:

unless ENV['NO_VAGRANT_APTGET']
  config.vm.provision :shell, :inline => '/usr/bin/apt-get update'
end

and then run “vagrant provision” as follows:

NO_VAGRANT_APTGET=1 vagrant provision

Dropping Tarballs with Puppet

I frequently find myself using Puppet to expand tarballs in various locations, sometimes fiddling with a directory name here or there. In fact, I do it so often, that I created a “define” for it earlier this week. This could be a little more polished, but in the spirit of sharing first drafts, here goes:

# Small define to expand a tarball at a location; assumes File[$title]
# definition of tarball and installation of pax:

define baselayout::drop_tarball($dest, $dir_name, $dir_sub='') {

  # $dest: cwd in which expansion is done
  # $dir_name: name of top level directory created in $dest
  # $dir_sub: regexp to -s for pax - not supported for .zip archives

  if ($dir_sub) {
    $regexp = "-s $dir_sub"
  } else {
    $regexp = ''
  }

  # CentOS' pax doesn't support "-j" flag; therefore, run pax after
  # bzcat in a pipeline. Twiddle path to bzcat as distro-appropriate:
  case $operatingsystem {
    CentOS: {
      $bzcat = "/usr/bin/bzcat"
    }
    Ubuntu: {
      $bzcat = "/bin/bzcat"
    }
  }
  
  # Choose expansion method based on file suffix:
  if (($title =~ /.tar.gz$/) or ($title =~ /.tgz$/)) {
    $expand = "/usr/bin/pax -rz $regexp < $title"
  } elsif (($title =~ /.tar.bz2$/) or ($title =~ /.tbz$/)) {
    $expand = "$bzcat $title | /usr/bin/pax -r $regexp"
  } elsif ( $title =~ /.zip$/ ) {
    $expand = "/usr/bin/unzip $title"
  }
  
  exec { "drop_tarball $title":
    command => $expand,
    cwd => $dest,
    creates => "${dest}/${dir_name}",
    require => File[$title],
  }
  
}

The definition is written for Ubuntu and CentOS, assumes pax is installed on the system, and that a file resource for the tarball is defined before the definition is called. Pax is used instead of tar to facilitate renaming the top-level directory of the tarball. Zipped directories are also support, but without rename functionality.

I’ll update a gist as I develop the definition.

Comments welcome.