Installing user gems using chef

In my experience installing server infrastructure using Chef, I usually use the chef_gem resource to install a gem that's needed in order to orchestrate the setup process. These are gems that are consumed by chef, or a chef resource to converge a node. However last night I was editing a chef recipe that's included in a chef_workstation cookbook that my team at CenturyLink Cloud uses for provisioning developer vagrant boxes and our TeamCity build agents. I have bloged about that in more detail here. The recipe I was working on is responsible for installing all of the gems we use in our chef dev process. They include both publicly available knife plugins and internally authored tools as well. These gems are consumed by the user of the vagrant box and not chef directly.

The bash force approach

This was one of the first cookbooks we created when we had limited knowledge of chef, ruby and how gems worked in general - alas our knowledge still has limits but they are much less restrictive. So this recipe looked something like this:

bash "install chef gems" do
  code <<-EOS
    su - #{node["chef_workstation"]["user"]} -c "chef gem install my-gem-1"
    su - #{node["chef_workstation"]["user"]} -c "chef gem install my-gem-2"
    su - #{node["chef_workstation"]["user"]} -c "chef gem install my-gem-3"
  EOS
end

As you can see we were just running this via bash. Really this is fine and it worked. There is alot to be said for something that works. When we were putting this cookbook together we found that using chef_gem or gem_package installed the gems into the root user's directory making them inaccesible to the vagrant user. So using su was our workaround.

Recently we started using Nexus as an internal gem repository that seems to have slightly different install behavior from rubygems or artifactory. The former repositories would always install the latest gem assuming we had no constraints. Nexus would not install anything if any version of the requested gem was already installed. This did not work well for our internal gem workflow where we expect a vagrant provision to always install the latest gem.

Using gem_package

My first thought was to use the raw ruby gem modules to check for updates, install if the gem was missing or run an update if there was a newer version. That could have worked but it just seemed like I must be reinventing a wheel so I revisited the gem_package docs. Not sure why, but I didn't find anything meeting my needs on StackOverflow or the other google results I was turning up.

After reviewing the docs and a couple of initial failed attempts I landed on the right attribute values that yielded what worked:

%w[clc-gem1 clc-gem2 clc_gem-amazing].each do |gem|
  gem_package gem do
    source node["clc_nexus"]["repo"]["localgems"]
    gem_binary "/opt/chefdk/embedded/bin/gem"
    options "--no-user-install"
    action :upgrade
  end
end

The key attributes here are gem_binary and options. Because I lean toward the idiot side of the intelligence spectrum, I had initially written off the gem_binary attribute thinking it was pointing to where to install binary gems. Nope, its intended to point to the location of the gem bin you want to use. Handy when you have multiple ruby installs. We use the chefdk on our vagrant boxes so that's where I point the gem_binary attribute.

The other not so obvious thing to add is the --no-user-install option. Since chef is running as root, if this is not specified, the gems are installed in /home/root. By specifying --no-user-install, the gems are installed in the shared ruby gems location. This may not be ideal and I'm sure there must be a way to get it in the vagrant user directory, but for the purposes of our vagrant environments, this works well.