02 Nov 2009
•
6 min read
Using the New Gem Bundler Today
As you might have heard, Carl and I released a new project that allows you to bundle your gems (both pure-ruby and native) with your application. Before I get into the process for using the bundler today, I'd like to go into the design goals of the project.
- The bundler should allow the specification of all dependencies in a separate place from the application itself. In other words, it should be possible to determine the dependencies for an application without needing to start up the application.
- The bundler should have a built-in dependency resolving mechanism, so it can determine the required gems for an entire set of dependencies.
- Once the dependencies are resolved, it should be possible to get the application up and running on a new system without needing to check Rubyforge (or gemcutter) again. This is especially important for compiled gems (it should be possible to get the list of required gems once and compile on remote systems as desired).
- Above all else, the bundler should provide a reproducible installation of Ruby applications. New gem releases or down remote servers should not be able to impact the successful installation of an application. In most cases,
git clone; gem bundle
should be all that is needed to get an application on a new system and up and running. - Finally, the bundler should not assume anything about Rails applications. While it should work flawlessly in the context of a Rails application, this should be because a Rails application is a Ruby application.
Using the Bundler Today
To use the gem bundler today in a non-Rails application, follow the following steps:gem install bundler
- Create a
Gemfile
in the root of your application - Add dependencies to your Gemfile. See below for more details on the sorts of things you can do in the Gemfile. At the simplest level,
gem "gem_name", "version"
will add a dependency of the gem and version to your application - At the root, run
gem bundle
. The bundler should tell you that it is resolving dependencies, then downloading and installing the gems. - Add
vendor/gems/gems
,vendor/gems/specifications
,vendor/gems/doc
, andvendor/gems/environment.rb
to your.gitignore file
- Inside your application, require
vendor/gems/environment.rb
to add the bundled dependencies to your load paths. - Use
Bundler.require_env :optional_environment
to actually require the files. - After committing, run
gem bundle
in a fresh clone to re-expand the gems. Since you left thevendor/gems/cache
in your source control, new machines will be guaranteed to use the same files as the original machine, requiring no remote dependencies
bin
directory. You can, therefore, run bin/rackup
for instance, which will ensure that the local bundle, rather than the system, is used. You can also run gem exec rackup
, which runs any command using the local bundle. This allows things like gem exec ruby -e "puts Nokogiri::VERSION"
or the even more adventurous gem exec bash
, which will open a new shell in the context of the bundle.
Gemfile
You can do any of the following in the Gemfile:gem "name", "version"
:version
may be a strict version or a version requirement like>= 1.0.6
. The version is optional.gem "name", "version", :require_as => "file"
: therequire_as
allows you to specify which file should be required when therequire_env
is called. By default, it is the gem's namegem "name", "version", :only => :testing
: The environment name can be anything. It is used later in yourrequire_env
call. You may specify either:only
, or:except
constraintsgem "name", "version", :git => "git://github.com/wycats/thor"
: Specify a git repository to be used to satisfy the dependency. You must use a hard dependency ("1.0.6") rather than a soft dependency (">= 1.0.6"). If a .gemspec is found in the repository, it is used for further dependency lookup. If the repository has multiple .gemspecs, each directory will a .gemspec will be considered a gem.gem "name", "version", :git => "git://github.com/wycats/thor", :branch => "experimental"
: Further specify a branch, tag, or ref to use. All of :branch, :tag, and :ref are valid optionsgem "name", "version", :vendored_at => "vendor/nokogiri"
: In the next version of bundler, this option will be changing to:path
. This specifies that the dependency can be found in the local file system, rather than remotely. It is resolved relative to the location of the Gemfileclear_sources
: Empties the list of gem sources to search inside of.source "http://gems.github.com"
: Adds a gem source to the list of available gem sources.bundle_path "vendor/my_gems"
: Changes the default location of bundled gems fromvendor/gems
bin_path "my_executables"
: Changes the default location of the installed executablesdisable_system_gems
: Without this command, both bundled gems and system gems will be used. You can therefore have things likeruby-debug
in your system and use it. However, it also means that you may be using something in development mode that is installed on your system but not available in production. For this reason, it is best todisable_system_gems
disable_rubygems
: This completely disables rubygems, reducing startup times considerably. However, it often doesn't work if libraries you are using depend on features of Rubygems. In this mode, the bundler shims the features of Rubygems that we know people are using, but it's possible that someone is using a feature we're unaware of. You are free to trydisable_rubygems
first, then remove it if it doesn't work. Note that Rails 2.3 cannot be made to work in this modeonly :environment { gem "rails" }
: You can useonly
orexcept
in block mode to specify a number of gems at once
Bundler process
When you rungem bundle
, a few things happen. First, the bundler attempts to resolve your list of dependencies against the gems you have already bundled. If they don't resolve, the metadata for each specified source is fetched and the gems are downloaded. Next (either way), the bundler checks to see whether the downloaded gems are expanded. For any gem that is not yet expanded, the bundler expands it. Finally, the bundler creates the environment.rb
file with the new settings. This means that running gem bundler
over and over again will be extremely fast, because after the first time, all gems are downloaded and expanded. If you change settings, like disable_rubygems
, running gem bundle
again will simply regenerate the environment.rb
.
Rails 2.3
To get this working with Rails 2.3, you need to create apreinitializer.rb
and insert the following:
```ruby
require "#{File.dirname(__FILE__)}/../vendor/bundler_gems/environment"
class Rails::Boot
def run
load_initializer
extend_environment
Rails::Initializer.run(:set_load_path)
end
def extend_environment
Rails::Initializer.class_eval do
old_load = instance_method(:load_environment)
define_method(:load_environment) do
Bundler.require_env RAILS_ENV
old_load.bind(self).call
end
end
end
end
It's a bit ugly, but you can copy and paste that code and forget it. Astute readers will notice that we're using <code>vendor/bundler_gems/environment.rb</code>. This is because Rails 2.3 attaches special, irrevocable meaning to <code>vendor/gems</code>. As a result, make sure to do the following in your <code>Gemfile</code>: <code>bundle_path "vendor/bundler_gems"</code>.
Gemcutter uses this setup and it's working great for them.
<h3>Bundler 0.7</h3>
We're going to be releasing Bundler 0.7 tomorrow. It has some new features:
<ul>
<li>List outdated gems by passing <code>--outdated-gems</code>. Bundler conservatively does not update your gems simply because a new version came out that satisfies the requirement. This is so that you can be sure that the versions running on your local machine will make it safely to production. This will allow you to check for outdated gems so you can decide whether to update your gems with --update. Hat tip to <code>manfred</code>, who submitted this patch as part of his submission to the <a href="http://2009.rubyenrails.nl/">Rumble at Ruby en Rails</a></li>
<li>Specify the build requirements for gems in a YAML file that you specify with <code>--build-options</code>. The file looks something like this:
```yaml
mysql:
config: /path/to/mysql_config
This is equivalent to --with-mysql-config=/path/to/mysql_config
:bundle => false
to indicate that you want to use system gems for a particular dependency. This will ensure that it gets resolved correctly during dependency resolution but does not need to be included in the bundle:vendored_at
and replace with :path
directory
DSL method in the Gemfile
:
directory "vendor/rails" do
gem "activesupport", :path => "activesupport" # :path is optional if it's identical to the gem name
# the version is optional if it can be determined from
# a gemspec
end
git
DSL method
```ruby
git "git://github.com/rails/rails.git" do # :branch, :tag, or :ref work here
gem "activesupport", :path => "activesupport" # same rules as directory, except that the files are
# first downloaded from git.
end
```