6 min read

Using the New Gem Bundler Today

As you might have heard, Carl and I released a new project that allows you to bundle your gems (both pure-ruby and native) with your application. Before I get into the process for using the bundler today, I'd like to go into the design goals of the project.

  • The bundler should allow the specification of all dependencies in a separate place from the application itself. In other words, it should be possible to determine the dependencies for an application without needing to start up the application.
  • The bundler should have a built-in dependency resolving mechanism, so it can determine the required gems for an entire set of dependencies.
  • Once the dependencies are resolved, it should be possible to get the application up and running on a new system without needing to check Rubyforge (or gemcutter) again. This is especially important for compiled gems (it should be possible to get the list of required gems once and compile on remote systems as desired).
  • Above all else, the bundler should provide a reproducible installation of Ruby applications. New gem releases or down remote servers should not be able to impact the successful installation of an application. In most cases, git clone; gem bundle should be all that is needed to get an application on a new system and up and running.
  • Finally, the bundler should not assume anything about Rails applications. While it should work flawlessly in the context of a Rails application, this should be because a Rails application is a Ruby application.

Using the Bundler Today

To use the gem bundler today in a non-Rails application, follow the following steps:
  1. gem install bundler
  2. Create a Gemfile in the root of your application
  3. Add dependencies to your Gemfile. See below for more details on the sorts of things you can do in the Gemfile. At the simplest level, gem "gem_name", "version" will add a dependency of the gem and version to your application
  4. At the root, run gem bundle. The bundler should tell you that it is resolving dependencies, then downloading and installing the gems.
  5. Add vendor/gems/gems, vendor/gems/specifications, vendor/gems/doc, and vendor/gems/environment.rb to your .gitignore file
  6. Inside your application, require vendor/gems/environment.rb to add the bundled dependencies to your load paths.
  7. Use Bundler.require_env :optional_environment to actually require the files.
  8. After committing, run gem bundle in a fresh clone to re-expand the gems. Since you left the vendor/gems/cache in your source control, new machines will be guaranteed to use the same files as the original machine, requiring no remote dependencies
The bundler will also install binaries into the app's bin directory. You can, therefore, run bin/rackup for instance, which will ensure that the local bundle, rather than the system, is used. You can also run gem exec rackup, which runs any command using the local bundle. This allows things like gem exec ruby -e "puts Nokogiri::VERSION" or the even more adventurous gem exec bash, which will open a new shell in the context of the bundle.

Gemfile

You can do any of the following in the Gemfile:
  • gem "name", "version": version may be a strict version or a version requirement like >= 1.0.6. The version is optional.
  • gem "name", "version", :require_as => "file": the require_as allows you to specify which file should be required when the require_env is called. By default, it is the gem's name
  • gem "name", "version", :only => :testing: The environment name can be anything. It is used later in your require_env call. You may specify either :only, or :except constraints
  • gem "name", "version", :git => "git://github.com/wycats/thor": Specify a git repository to be used to satisfy the dependency. You must use a hard dependency ("1.0.6") rather than a soft dependency (">= 1.0.6"). If a .gemspec is found in the repository, it is used for further dependency lookup. If the repository has multiple .gemspecs, each directory will a .gemspec will be considered a gem.
  • gem "name", "version", :git => "git://github.com/wycats/thor", :branch => "experimental": Further specify a branch, tag, or ref to use. All of :branch, :tag, and :ref are valid options
  • gem "name", "version", :vendored_at => "vendor/nokogiri": In the next version of bundler, this option will be changing to :path. This specifies that the dependency can be found in the local file system, rather than remotely. It is resolved relative to the location of the Gemfile
  • clear_sources: Empties the list of gem sources to search inside of.
  • source "http://gems.github.com": Adds a gem source to the list of available gem sources.
  • bundle_path "vendor/my_gems": Changes the default location of bundled gems from vendor/gems
  • bin_path "my_executables": Changes the default location of the installed executables
  • disable_system_gems: Without this command, both bundled gems and system gems will be used. You can therefore have things like ruby-debug in your system and use it. However, it also means that you may be using something in development mode that is installed on your system but not available in production. For this reason, it is best to disable_system_gems
  • disable_rubygems: This completely disables rubygems, reducing startup times considerably. However, it often doesn't work if libraries you are using depend on features of Rubygems. In this mode, the bundler shims the features of Rubygems that we know people are using, but it's possible that someone is using a feature we're unaware of. You are free to try disable_rubygems first, then remove it if it doesn't work. Note that Rails 2.3 cannot be made to work in this mode
  • only :environment { gem "rails" }: You can use only or except in block mode to specify a number of gems at once

Bundler process

When you run gem bundle, a few things happen. First, the bundler attempts to resolve your list of dependencies against the gems you have already bundled. If they don't resolve, the metadata for each specified source is fetched and the gems are downloaded. Next (either way), the bundler checks to see whether the downloaded gems are expanded. For any gem that is not yet expanded, the bundler expands it. Finally, the bundler creates the environment.rb file with the new settings. This means that running gem bundler over and over again will be extremely fast, because after the first time, all gems are downloaded and expanded. If you change settings, like disable_rubygems, running gem bundle again will simply regenerate the environment.rb.

Rails 2.3

To get this working with Rails 2.3, you need to create a preinitializer.rb and insert the following: ```ruby require "#{File.dirname(__FILE__)}/../vendor/bundler_gems/environment"

class Rails::Boot
def run
load_initializer
extend_environment
Rails::Initializer.run(:set_load_path)
end

def extend_environment
Rails::Initializer.class_eval do
old_load = instance_method(:load_environment)
define_method(:load_environment) do
Bundler.require_env RAILS_ENV
old_load.bind(self).call
end
end
end
end

It's a bit ugly, but you can copy and paste that code and forget it. Astute readers will notice that we're using <code>vendor/bundler_gems/environment.rb</code>. This is because Rails 2.3 attaches special, irrevocable meaning to <code>vendor/gems</code>. As a result, make sure to do the following in your <code>Gemfile</code>: <code>bundle_path "vendor/bundler_gems"</code>.

Gemcutter uses this setup and it's working great for them.
<h3>Bundler 0.7</h3>
We're going to be releasing Bundler 0.7 tomorrow. It has some new features:
<ul>
	<li>List outdated gems by passing <code>--outdated-gems</code>. Bundler conservatively does not update your gems simply because a new version came out that satisfies the requirement. This is so that you can be sure that the versions running on your local machine will make it safely to production. This will allow you to check for outdated gems so you can decide whether to update your gems with --update. Hat tip to <code>manfred</code>, who submitted this patch as part of his submission to the <a href="http://2009.rubyenrails.nl/">Rumble at Ruby en Rails</a></li>
	<li>Specify the build requirements for gems in a YAML file that you specify with <code>--build-options</code>. The file looks something like this:
```yaml
mysql:
  config: /path/to/mysql_config

This is equivalent to --with-mysql-config=/path/to/mysql_config

  • Specify :bundle => false to indicate that you want to use system gems for a particular dependency. This will ensure that it gets resolved correctly during dependency resolution but does not need to be included in the bundle

  • Support for multiple bundles containing multiple platforms. This is especially useful for people moving back and forth between Ruby 1.8 and 1.9 and don't want to constantly have to nuke and rebundle

  • Deprecate :vendored_at and replace with :path

  • A new directory DSL method in the Gemfile:

    directory "vendor/rails" do
      gem "activesupport", :path =&gt; "activesupport" # :path is optional if it's identical to the gem name
                                                    # the version is optional if it can be determined from
                                                    # a gemspec
    end
    
  • You can do the same with the git DSL method ```ruby git "git://github.com/rails/rails.git" do # :branch, :tag, or :ref work here gem "activesupport", :path => "activesupport" # same rules as directory, except that the files are # first downloaded from git. end ```
  • Fix some bugs in resolving prerelease dependencies