Yehuda Katz is a member of the Ember.js, Ruby on Rails and jQuery Core Teams; his 9-to-5 home is at the startup he founded, Tilde Inc.. There he works on Skylight, the smart profiler for Rails, and does Ember.js consulting. He is best known for his open source work, which also includes Thor and Handlebars. He travels the world doing open source evangelism and web standards work.

Archive for the ‘Other’ Category

Ruby is NOT a Callable Oriented Language (It’s Object Oriented)

I recently ran across a presentation entitled Python vs. Ruby: A Battle to the Death. I didn’t consider it to be a particularly fair battle, and may well reply in more detail in a later post.

However, what struck me as most worthy of explanation was the presenter’s concern about the fact that Procs are not callable via parens.

x = Proc.new { puts "HELLO" }
x() #=> undefined method `x' for #<Object:0x1001bd298><
x.call #=> "HELLO"
x[]    #=> "HELLO"

For those coming from a callable-oriented language, like Python, this seems horribly inconsistent. Why are methods called with (), while Procs are called with [].

But what’s going on here is that Ruby doesn’t have a notion of “callable”, like Python. Instead, it has a pervasive notion of Object. Here, x is an instance of the Proc class. Both call and [] are methods on the Proc class.

Designing for the Common Case: Calling Methods

Coming from a callable-oriented language, this might seem jarring. But Ruby is designed around Objects and the common cases of working with objects.

Calling methods is far more common than wanting to get an instance of Method, so Ruby optimizes the case of calling methods, with a slightly less elegant form to access an instance:

class Greeter
  def say
    puts "Hello world!"
  end
end
 
Greeter.new.say #=> "Hello world!"
 
Greeter.method(:new) #=> #<Method: Class#new>
Greeter.new.method(:say) #=> #<Method: Greeter#say>
 
# This is so that you don't have to say:
 
Greeter.new().say()

Ruby considers the common case of calling methods, and optimizes that case while still making the less common case possible.

Designing for the Common Case: How Blocks Are Really Used

One of the reasons that the Proc.new case throws off Rubyists in debates is that Rubyists literally never call Proc objects using [].

In Ruby, Procs are the object passed to methods when using block syntax. Here is how Procs are actually used:

def hello
  puts "Hello world!"
  yield
  puts "Goodbye cruel world!"
end
 
hello { puts "I am in the world!" }

When examining languages that support passing anonymous functions to functions (like JavaScript), it turns out that the vast majority of such cases involve a single anonymous function. As a result, Matz (inspired by Smalltalk) built in the idea of a block as a core construct in Ruby. In the vast majority of cases, blocks are created using lightweight syntax ({} or do/end) and called using yield.

In some cases, blocks are passed from one method to the next, before they are finally called using yield:

def step1(&block)
  puts "Step 1"
  step2(&block)
end
 
def step2
  puts "Step 2"
  yield
end
 
step1 { puts "Do the action!" } #=> "Step 1\nStep 2\nDo the action!"

As you can see, Ruby builds in the idea of calling a block into the core language. I searched through Rails (a fairly large codebase) for instances of using [] to call a Proc and while we use blocks extremely commonly, we don’t use [] to call them.

I suspect that the reason this comes up is that people who are used to having to define standalone functions, pass them around, and then call them are looking for the analogous constructs in Ruby, but are missing the different paradigm used by Ruby.

Consistent Method Execution

Fundamentally, the issue here comes down to this:

def foo
  proc {}
end
 
foo()

In Ruby, methods are invoked with our without parentheses. All methods return values, which are always Objects. All Objects have methods. So foo() is a method call that returns a Proc object. It’s extremely consistent, with very few axioms. The fact that the axioms aren’t the same as those in a callable-oriented language doesn’t make them “weird”.

AbstractQueryFactoryFactories and alias_method_chain: The Ruby Way

In the past week, I read a couple of posts that made me really want to respond with a coherent explanation of how I build modular Ruby code.

The first post, by Nick Kallen of Twitter, gushed about the benefits of PerQueryTimingOutQueryFactory and called out Ruby (and a slew of other “hipster” languages) for using language features (like my “favorite” alias_method_chain) and leveraging dynamicism to solve problems that he argues are more appropriately solved with laugh-inducing pattern names:

In a very dynamic language like Ruby, open classes and method aliasing (e.g., alias_method_chain) mitigate this problem, but they don’t solve it. If you manipulate a class to add logging, all instances of that class will have logging; you can’t take a surgical approach and say “just objects instantiated in this context”.

If you haven’t read it yet, you should probably read it now (at least skim it).

As if on cue, a post by Pivot Rob Olson demonstrated the lengths some Rubyists will go to torture alias_method_chain to solve essentially the same problem that Nick addressed.

In short, while I agree in principle with Nick, his examples and the jargon he used demonstrated exactly why so few Rubyists take his point seriously. It is possible to write modular code in Ruby with the same level of flexibility but with far less code and fewer concept hoops to jump through.

Let’s take a look at the problem Rob was trying to solve:

module Teacher
  def initialize
    puts "initializing teacher"
  end
end
 
class Person
  include Teacher
 
  def initialize
    puts "initializing person"
  end
end
 
# Desired output:
# > Person.new
# initializing teacher
# initializing person

This is a classic problem involving modularity. In essence, Rob wants to be able to “decorate” the Person class to include teacher traits.

Nick’s response would have been to create a factory that creates a Person proxy decorated with Teacher properties. And he would have been technically correct, but that description obscures the Ruby implementation, and makes it sound like we need new “Factory” and “Decorator” objects, as we do, in fact, need when programming in Java.

In Ruby, you’d solve this problem thusly:

# The base person implementation. Never instantiate this.
# Instead, create a subclass that mixes in appropriate modules.
class AbstractPerson
  def initialize
    puts "Initializing person"
  end
end
 
# Provide additional "teacher" functionality as a module. This can be
# mixed into subclasses of AbstractPerson, giving super access to
# methods on AbstractPerson
module Teacher
  def initialize
    puts "Initializing teacher"
    super
  end
end
 
# Our actual Person class. Mix in whatever modules you want to
# add new functionality.
class Person < AbstractPerson
  include Teacher
end
 
# > Person.new
# Initializing teacher
# Initializing person

Including modules essentially decorates existing classes with additional functionality. You can include multiple modules to layer on existing functionality, but you don’t need to create special factory or decorator objects to make this work.

For those following along, the classes used here are “factories”, and the modules are “decorators”. But just as it’s not useful to constantly think about classes as “structs with function pointers” because that’s historically how they were implemented, I’d argue it’s not useful to constantly think about classes and modules as factories and decorators, simply because they’re analogous to those concepts in languages like Java.

The Case of the PerQueryTimingFactoryFactory

Nick’s example is actually a great example of a case where modularity is important. In this case, he has a base Query class that he wants to extend to add support for timeouts. He wrote his solution in Scala; I’ll transcode it into Ruby.

Feel free to skim the examples that follow. I’m transcoding the Scala into Ruby to demonstrate something which you will be able to understand without fully understanding the examples.

class QueryProxy
  def initialize(query)
    @query = query
  end
 
  def select
    delegate { @query.select { yield } }
  end
 
  def execute
    delegate { @query.execute }
  end
 
  def cancel
    @query.cancel
  end  
 
  def delegate
    yield
  end
end

Then, in order to add support for Timeouts, he creates a new subclass of QueryProxy:

class TimingOutQuery < QueryProxy
  def initialize(query, timeout)
    @timeout = timeout
    @query   = query
  end
 
  def delegate
    begin
      Timeout.timeout(@timeout) do
        yield
      end
    rescue Timeout::Error
      cancel
      raise SqlTimeoutException
    end
  end
end

Next, in order to instantiate a TimingOutQuery, he creates a TimingOutQueryFactory:

class TimingOutQueryFactory
  def initialize(query_factory, timeout)
    @query_factory = query_factory
    @timeout = timeout
  end
 
  def self.call(connection, query, *args)
    TimingOutQuery.new(@query_factory.call(connection, query, *args), timeout)
  end
end

As his coup de grâce, he shows how, now that everything is so modular, it is trivial to extend this system to support timeouts that were per-query.

class PerQueryTimingOutQueryFactory
  def initialize(query_factory, timeouts)
    @query_factory = query_factory
    @timeouts = timeouts
  end
 
  def self.call(connection, query, *args)
    TimingOutQuery.new(@query_factory.call(connection, query, *args), @timeouts[query])
  end
end

This is all true. By using factories and proxies, as you would in Java, this Ruby code is modular. It is possible to create a new kind of QueryFactory trivially.

However, this code, by tacking close to vocabulary created to describe Java patterns, rebuilds functionality that exists natively in Ruby. It would be equivalent to creating a Hash of Procs in Ruby when a Class would do.

The Case: Solved

Ruby natively provides factories, proxies and decorators via language features. In fact, that vocabulary obscures the obvious solution to Nick’s problem.

# No need for a proxy at all, so we skip it
 
module Timeout
  # super allows us to delegate to the Query this
  # module is included into, even inside a block
  def select
    timeout { super }
  end
 
  def execute
    timeout { super }
  end
 
  # We get the cancel delegation natively, because
  # we can use subclasses, rather than separate
  # proxy object, to implement the proxy
 
private
  # Since we're not using a proxy, we'll just implement
  # the timeout method directly, and skip "delegate"
  def timeout
    # The Timeout module expects a duration method
    # which classes that include Timeout should provide
    Timeout.timeout(duration) do
      yield
    end
  rescue Timeout::Error
    cancel
    raise SqlTimeoutException
  end
end
 
# Classes in Ruby serve double duty as "proxies" and
# "factories". This behavior is part of Ruby semantics.
class TimingOutQuery < Query
  include Timeout
 
private
  # implement duration to hardcode the value of 1
  def duration
    1
  end
end
 
# Creating a second subclass of Query, this time with
# per-query timeout semantics.
class PerQueryTimingOutQuery < Query
  TIMEOUTS = Hash.new(0.5).merge("query1" => 1, "query2" => 3)
 
  include Timeout
 
private
  def duration
    TIMEOUTS[query]
  end
end

As Nick would point out, what we’re doing here, from a very abstract perspective, isn’t all that different from his example. Our subclasses are proxies, our modules are decorators, and our classes are serving as factories. However, forcing that verbiage on built-in Ruby language features, in my opinion, only serves to complicate matters. More importantly, by starting to think about the problem in terms of the Java-inspired patterns, it’s easy to end up building code that looks more like Nick’s example than my solution above.

For the record, I think that designing modularly is very important, and while Ruby provides built-in support for these modular patterns, we don’t see enough usage of them. However, we should not assume that the reason for the overuse of poor modularity patterns (like alias_method_chain) result from a lack of discussion around proxies, decorators, and factories.

By the way, ActionController in Rails 3 provides an abstract superclass called ActionController::Metal, a series of modules that users can mix in to subclasses however they like, and a pre-built ActionController::Base with all the modules mixed in (to provide the convenient “default” experience). Additionally, users or extensions can easily provide additional modules to mix in to ActionController::Metal subclasses. This is precisely the pattern I am describing here, and I strongly recommend that Rubyists use it more when writing code they wish to be modular.

Postscript: Scala

When researching for this article, I wondered why Nick hadn’t used Ruby’s equivalent to modules (traits) in his examples. It would be possible to write Scala code that was extremely similar to my preferred solution to the problem. I asked both Nick and the guys in #scala. Both said that while traits could solve this problem in Scala, they could not be used flexibly enough at runtime.

In particular, Nick wanted to be able to read the list of “decorators” to use at runtime, and compose something that could create queries with the appropriate elements. According to the guys in #scala, it’s a well-understood issue, and Kevin Wright has a compiler plugin to solve this exact problem.

Finally, the guys there seemed to generally agree with my central thesis: that thinking about problems in terms of patterns originally devised for Java can leave a better, more implementation-appropriate solution sitting on the table, even when the better solution can be thought of in terms of the older pattern (with some contortions).

The Blind Men and the Elephant: A Story of Noobs

If you will indulge me, I’d like to paraphrase a familiar tale:

Once upon a time, deep in the forest, there was a tribe of elephant curators. The elders of this tribe kept sophisticated, detailed notes about the proper care and feeding of elephants, and the villagers tended to follow along.

Eventually, they dedicated a large section of the local library to books and articles on the care and feeding of elephants.

One day, a group of blind nomads appeared in the village. Each of the blind men went to greet the villagers, and were met with welcomes. Wanting to be helpful, they walked over to one of the elephants and tried to learn about it.

The first man, who stood next to the elephant’s tail said, “I feel a snake”.

The second, who stood next to the elephant’s leg said, “I feel a tree trunk”.

And so on.

One of the group of nomads, who thought he felt a snake, went to the elders of the village and asked, “I would like to help. How can I feed this creature?”

The elder replied: “Sir, if you can’t be bothered to search the library for information on the care and feeding of elephants, surely you are wasting our time”.

Years passed and after a grueling series of trials, the blind men became integrated into the culture, becoming some of the most successful at caring for the elephants.

Eventually, another group of blind nomads appeared. The entire village, including the original group, proceeded to berate the new travelers. “We’ve spent quite a bit of time putting together a section of the library about how to care for these animals. You are wasting our time”.

Of course, the travelers did not know the creatures were elephants, and so they entered the library, searching for books on feeding snakes and caring for trees.

Eventually, an elderly blind man, of the original group stood up and said: “Have we forgotten than we, too, started in this confused state. We should help these travelers and perhaps they will become as wise and helpful as we became”.

To many participants in open source communities, this is a familiar tale. When a developer first comes across an open source project, either to use it in a project or to help, he is like a blind man feeling an elephant.

It’s easy to spit out “lmgtfy.com” or RTFM, but in truth, these beginners barely know where to look. All too often, we (open source leaders) assume that if someone couldn’t figure out the right search term on Google, they can never become a viable community member.

When I first started working on Rails, I distinctly remember not knowing what the request method in Rails controllers was. To some degree, this could be attributed to its exclusion in api.rubyonrails.org, but some judicious Googling turned up the ActionController::Request class. Writing this post years later, the request method still does not reside in the API docs, but I found the Request documentation in seconds.

The problem is that a new developer simply has no conceptual model for the problem at all. In most cases, the “noob” can stare at “the f***ing manual” all day and simply fail to find something staring him in the face. Importantly, this does not reflect a failing on the part of the new developer. Virtually everyone I know who worked their way from noob to senior Rails developer starting feeling around the elephant.

As open source leaders, if we are interested in growing our communities, we should treat new developers as confused people with real potential. That’s not to say that sinking dozens of hours down a black hole is a good use of time. On the other hand, the mismatch between how we think about problems once we become experienced and the way we feel around like a blind man when getting started makes the experience of getting started with an open source project far more painful than it needs to be.

Using Bundler in Real Life

A lot of people have asked me what the recommended workflows for bundler are. Turns out, they’re quite simple.

Let’s step through a few use-cases.

You Get a Repo for the First Time

You’ve just checked out a git (or other) repository for an application that uses bundler. Regardless of any other features of bundler in use, just run:

bundle install

This will resolve all dependencies and install the ones that aren’t already installed in system gems or in your system’s bundler cache.

You Update a Repo Using Bundler

If you update a repository using bundler, and it has updated its dependencies in the Gemfile, regardless of any other features in use, just run:

bundle install

As above, this will resolve all dependencies and install any gems that are not already installed.

You have created a new Rails application

If you’ve created a new Rails application, go inside it and run:

bundle install

This will make sure that Rails’ dependencies (such as SQLite3) are available in system gems or your system’s bundler cache. In most cases, this will not need to install anything.

To check whether your system already satisfied the application’s dependencies, run:

bundle check

As you work in the application, you may wish to add dependencies. To do so, update the Gemfile, and run:

bundle install

You are ready to share your new Rails application

Now that you have a working Rails application, and wish to share, you might wish to ensure that your collaborators get the same versions of the gems as you have. For instance, if webrat specified “nokogiri >= 1.4″ as a dependency, you might want to ensure that an update to nokogiri does not change the actual gems that bundle install will install.

To achieve this, run:

bundle lock

This will create a new file called Gemfile.lock in your root directory that contains the dependencies that you specified, as well as the fully resolved dependency graph. Check that file into source control.

When your collaborators receive the repository and run bundle install, bundler will use the resolved dependencies in Gemfile.lock.

You have a locked application, and wish to add a new dependency

If you add a new dependency to your Gemfile in a locked application, Bundler will give you an error if you try to perform any operations.

You will want to run bundle unlock to remove the lock, then bundle install to ensure that the new dependencies are installed on your system, and finally bundle lock again to relock your application to the new dependencies.

We will add a command in a near-future version to perform all these steps for you (something like bundle install –relock).

You want a self-contained application

In many cases, it is desirable to be able to have a self-contained application that you can share with others which contains all of the required gems.

In addition to a general desire to remove a dependency on Gemcutter, you might have dependencies on gems that are not on a publicly accessible gem repository.

To collect up all gems and place them into your app, run:

bundle pack

When running bundle install in the future, Bundler will use packed gems, if available, in preference to gems available in other sources.

Conclusion

I hope these workflows have clarified the intent of Bundler 0.9 (and 1.0). During our work on earlier versions, the lack of these workflows came up again and again as a source of frustration. This was the primary reason for the big changes in 0.9, so I hope you find them useful.

The Building Blocks of Ruby

When showing off cool features of Ruby to the uninitiated (or to a language sparring partner), the excited Rubyist often shows off Ruby’s “powerful block syntax”. Unfortunately, the Rubyist uses “powerful block syntax” as shorthand for a number of features that the Pythonista or Javaist simply has no context for.

To start, we usually point at Rake, Rspec or Sinatra as examples of awesome usage of block syntax:

get "/hello" do
  "Hello World"
end

In response, Pythonistas usually point to these syntaxes as roughly equivalent:

@get('/hi')
def hello():
  return "Hello World"
 
def hello() -> "/hi":
  return "Hello World"

While the Python version may not be quite as pretty, nothing about them screams “Ruby has much stronger capabilities here”. Instead, by using examples like Sinatra, Rubyists trade in an argument about great semantic power for one about superficial beauty.

Rubyists, Pythonistas and others working on web development share a common language in JavaScript. When describing blocks to “outsiders” who share a common knowledge of JavaScript, we tend to point at JavaScript functions as a close analogue. Unfortunately, this only furthers the confusion.

On the Ruby side, when PHP or Java announces that they’re “adding closures”, many of us don’t stop to ask “what kind of closures?”

Cut to the Chase

Let’s cut to the chase and use a better example of the utility of Ruby blocks.

def append(location, data)
  path = Pathname.new(location)
  raise "Location does not exist" unless path.exist?
 
  File.open(path, "a") do |file|
    file.puts YAML.dump(data)
  end
 
  return data
end

Here, the File.open method takes a block. It then opens a new file (in “append” mode), and yields the open file into the block. When the block completes, Ruby closes the file. Except that Ruby doesn’t just close the file when the block completes; it guarantees that the File will be closed, even if executing the block results in a raise. Let’s take a look at the implementation of File in Rubinius:

def self.open(*args)
  io = new *args
 
  return io unless block_given?
 
  begin
    yield io
  ensure
    begin
      io.close unless io.closed?
    rescue StandardError
      # nothing, just swallow them.
    end
  end
end

This means that you can wrap up idioms like pervasive try/catch/finally in methods.

# Without blocks
def append(location, data)
  path = Pathname.new(location)
  raise "Location does not exist" unless path.exist?
 
  begin
    file = File.open(path, "a")
    file.puts YAML.dump(data)
  ensure
    file.close
  end
 
  return data
end

Because Ruby runs ensure clauses even when the exception happened in a block, programmers can reliably ensure that Ruby executes teardown logic hidden away in abstractions.

This example only demonstrates the power of well-designed lambdas. With the addition of one small additional feature, Ruby’s blocks become something altogether different.

def write(location, data)
  path = Pathname.new(location)
  raise "Location does not exist" unless path.exist?
 
  File.open(path, "w") do |file|
    return false if Digest::MD5.hexdigest(file.read) == data.hash
    file.puts YAML.dump(data)
  end
 
  return true
end

In the above case, imagine that writing the data to disk is quite expensive, and we can skip writing if the MD5 hash of the file’s contents match a hash method on the data. Here, we’ll return false if the method did not write to disk, and true if the method did.

Ruby’s blocks support non-local-return (some references), which means that a return from the block behaves identically to returning from the block’s original context. In this case, returning from inside the block returns from the write method, but Ruby will still run the ensure block closing the file.

You can think of non-local-return as behaving something like:

def write(location, data)
  path = Pathname.new(location)
  raise "Location does not exist" unless path.exist?
 
  File.open(path, "w") do |file|
    raise Return.new(false) if Digest::MD5.hexdigest(file.read) == data.hash
    file.puts YAML.dump(data)
  end
 
  return true
rescue Return => e
  return e.object
end

where Return is Return = Struct.new(:object).

Of course, any reasonable lambda implementation will support this, but Ruby’s version has the benefit of feeling just like a normal return, and requiring much less chrome to achieve it. It also behaves well in scenarios that already use rescue or ensure, avoiding mind-warping combinations.

Further, Ruby also supports super inside of blocks. Imagine the write method was defined on a subclass of a simpler class whose write method took the raw data from the file and printed it to a log.

def write(location, data)
  path = Pathname.new(location)
  raise "Location does not exist" unless path.exist?
 
  File.open(path, "w") do |file|
    file_data = file.read
    super(location, file_data)
    return false if Digest::MD5.hexdigest(file_data) == data.hash
    file.puts YAML.dump(data)
  end
 
  return true
end

In a purer lambda scenario, we would need to store off a reference to the self, then use that reference inside the lambda:

def write(location, data)
  path = Pathname.new(location)
  raise "Location does not exist" unless path.exist?
 
  this = self
  File.open(path, "w") do |file|
    file_data = file.read
 
    # imaginary Ruby construct that would be needed without
    # non-local-super
    this.super.write(location, file_data)
    raise Return.new(false) if Digest::MD5.hexdigest(file_data) == data.hash
    file.puts YAML.dump(data)
  end
 
  return true
rescue Return => e
  return e.object
end

You can also yield to a method’s block inside a block. Imagine that the write method is called with a block that chooses the correct data to use based on whether the file is executable:

def write(location)
  path = Pathname.new(location)
  raise "Location does not exist" unless path.exist?
 
  File.open(path, "w") do |file|
    file_data = file.read
    super(location)
    data = yield file
    return false if Digest::MD5.hexdigest(file_data) == data.hash
    file.puts YAML.dump(data)
  end
 
  return true
end

This would be called via:

write("/path/to/file") do |file|
  if file.executable?
    "#!/usr/bin/env ruby\nputs 'Hello World!'"
  else
    "Hello World!"
  end
end

In a pure-lambda language, we would take the block in as a normal argument to the function, then call it inside the closure:

def write(location, block)
  path = Pathname.new(location)
  raise "Location does not exist" unless path.exist?
 
  this = self
  File.open(path, "w") do |file|
    file_data = file.read
 
    # imaginary Ruby construct that would be needed without
    # non-local-super
    this.super.write(location, file_data)
    data = block.call(file)
    raise Return.new(false) if Digest::MD5.hexdigest(file_data) == data.hash
    file.puts YAML.dump(data)
  end
 
  return true
rescue Return => e
  return e.object
end

The real benefit of Ruby’s approach comes from the fact that the code inside the block would be identical if the method did not take a block. Consider the identical method, except taking a File instead of a location:

def write(file)
  file_data = file.read
  super(file)
  data = yield file
  return false if Digest::MD5.hexdigest(file_data) == data.hash
  file.puts YAML.dump(data)
  return true
end

Without the block, the Ruby code looks exactly the same. This means that Ruby programmers can more easily abstract out repeated patterns into methods that take blocks without having to rewrite a bunch of code. It also means that using a block does not interrupt the normal flow of code, and it’s possible to create new “control flow” constructs that behave almost identically to built-in control flow constructs like if and while.

Rails uses this to good effect with respond_to, which provides convenient syntax for declaring content negotiation:

def index
  @people = Person.find(:all)
 
  respond_to do |format|
    format.html # default action is render
    format.xml { render :xml => @people.xml }
  end
end

Because of the way Ruby blocks work, you can also return from any of the format blocks:

def index
  @people = Person.find(:all)
 
  respond_to do |format|
    format.html { redirect_to(person_path(@people.first)) and return }
    format.xml  { render :xml => @people.xml }
    format.json { render :json => @people.json }
  end
 
  session[:web_service] = true
end

Here, we returned from the HTML format after redirecting, allowing us to take additional action (setting a :web_service key on the session) for other cases (XML and JSON mime types).

Keep in mind that the code above is a demonstration of a number of features of Ruby’s blocks. It’s very rare to see return, yield and super all used in a single block. That said, Ruby programmers commonly use one or more of these constructs inside blocks, because their usage is seamless.

So Why Are Ruby’s Blocks Better?

If you made it this far, let’s take a look at another use of blocks in Ruby: mutex synchronization.

Java supports synchronization via a special synchronized keyword:

class Example {
  final Lock lock = new Lock();
 
  void example() {
    synchronized(lock) {
      // do dangerous stuff here
    }
  }
}

Essentially, Java provides a special construct for expressing the idea that it should run a block of code once at a time for a given instance of the synchronization object. Because Java provides a special construct, you can return from inside the synchronization block, and the Java runtime does the appropriate things.

Similarly, Python required the use of try/finally until Python 2.5, when they added a special language feature to handle the try/finally idiom:

class Example:
  # old
  def example(self):
    lock.acquire()
    try:
      ... access shared resource
    finally:
      lock.release() # release lock, no matter what
 
  # new
  def example(self):
    with lock:
      ... access shared resource

In Python’s 2.5′s case, the object passed to with must implement a special protocol (including __enter__ and __exit__ methods), so the with statement cannot be used like Ruby’s general-purpose, lightweight blocks.

Ruby represents the same concept using a method that takes a block:

class Example
  @@lock = Mutex.new
 
  def example
    @@lock.synchronize do
      # do dangerous stuff here
    end
  end
end

Importantly, synchronize is a normal Ruby method. The original version, written in pure Ruby, looks like this:

def synchronize
  lock
  begin
    yield
  ensure
    unlock
  end
end

It has all the hallmarks of what we’ve discussed so far. It locks, yields to the block, and ensures that the lock will be released. This means that if a Ruby programmer returns from inside the block, synchronize will behave correctly.

This example demonstrates the key power of Ruby’s blocks: they can easily replace language constructs. In this case, a Ruby programmer can take unsafe code, plop it inside a synchronization block, and it will continue to work.

Postscript

I’ve historically written my posts without very many links, mostly out of a fear of links going out of date. I’ve received increasing requests for more annotations in my posts, so I’ll start doing that. Let me know if you think my annotations in this post were useful, and feel free to give me any suggestions on that front that you find useful.

Bundler 0.9: Heading Toward 1.0

Over the past two years, Carl and I have been working on-again off-again on the problem of dependency resolution. Fabien Franzen implemented the first partial solution for Merb, with thor merb:gem:install.

When we started working on Rails, we knew we wanted to finally crack the nut, making it possible for Rails itself to have some of its own dependencies, and solving some persistent, long-term problems with gem plugins.

Earlier this year, we released bundler, our first real attempt to solve this problem. Among other things, it shipped with the first true dependency resolver for Rubygems. It also modeled the entire dependency process on top of Rubygems, allowing us to support git repositories in the same dependency resolution process as Rubygems themselves.

Over the next few months, we refined the bundler quite a bit. We’re proud of the fact that individuals, web shops, and deployment companies have adopted bundler, and Gemfile has become a standard way of expressing gem dependencies for an app.

During this time, we’ve also received quite a bit of feedback. As we approach the release of Bundler 1.0, which we hope to ship along with Rails 3.0 final, we took the opportunity to take a look at all the feedback people have sent so far.

Having done so, we’re proud to announce Bundler 0.9 with radically improved workflows that fit our needs and the needs of those who have contributed feedback and patches.

The Cache

Before Bundler 0.9, Bundler installed gems and git repositories to a local application cache. When we looked at the deployment workflow, we found that while people liked the ability to cache their .gem files in their application, they had a lot of trouble at deployment time.

Essentially, people (and we!) wanted the ability to expand the gems to a central cache that they could reuse across deployments. People tried various symlinking approaches, but it became clear to us Bundler 1.0 needed to support this workflow natively.

On a related note, people who developed a number of applications on a single machine wanted to be able to reuse a single system cache across their application, and not need to connect to remotes so often when they already had all the gems they needed on their systems.

Finally, Rails itself grated against the default Bundler workflow. Most people installing a new Rails app run gem install rails, then rails my_app, and then want to go into their application and start working. But with Bundler 0.5 to 0.8, they needed to hit the remotes again even though, by definition, the system gem cache already had all the gems they needed.

Runtime

The fully-packaged application works fantastically for deployment, enabling a repeatable, reliable development to staging to production workflow. However, the need to explicitly bundle the application after added a new dependency in the very early stages of an application’s life cycle feels more like a compile step than a lean mean agile machine.

In Bundler 0.9, we’re adding the ability to run your application against gems already on your system, with the same reliability that you’ve grown to love from our dependency resolver.

Once you’ve gotten your application working, you can lock down the dependencies, so coworkers or production will use exactly the same environment that worked for you.

This allows you to use your common system gem repository across multiple apps, with a fully resolved virtual environment for each application based on the app’s dependencies specified in the Gemfile.

Locking

A lot of people (including me) love the idea of storing the .gem files in the application to create a single deployable unit with no network dependencies.

Storing the .gem files in the repository also provides a record of the fully resolved dependencies for your application, so that a new released gem between testing and deployment cannot change the environment. In Bundler 0.9, we provide a mechanism for you to save that record without having to also store the .gem files.

This won’t give you a single, dependency-less deployment package, but it will save you from unexpected environment changes.

New Commands

When we started Bundler, we had just one command: gem bundle. Over the following months, the command took on a life of its own, with a slew of flags representing both options and verbs. Bundler 1.0 will have a small, streamlined list of commands, each with its own set of flags:

bundle pack

Take all .gem files needed for the application and place them into the local application. By default, bundler places them in vendor/gems.

bundle lock

Resolve all dependencies in the Gemfile, and store a record of that process. In the future, use that record, preventing any changes in the environment or in the remote sources from changing the gems used by the application.

bundle install

Install the bundle specified by the Gemfile to the cache. By default, Bundler installs the gems (and git repositories) into system gems. You can override this to install into another location, or into your local application, just like in Bundler 0.8 and before.

After cloning a new repository for the first time, you will probably want to run this command.

If you packed gems, Bundler will use the gems from your application. If you packed and locked gems, Bundler will install the gems immediately without updating the remote sources.

bundle check

Check to see whether your cache has all the required dependencies. If it does not, Bundler will print out the list of missing gems.

bundle exec …

Run the specified command using the local environment. For instance, if you have Rails 2.3 and 3.0 installed, if you run bundle exec rails . in a Rails 2.3 application, Bundler will run the rails command using Rails 2.3 (and other gems in the current environment).

Environments

In Bundler 0.9, as in Bundler 0.8 and before, the Gemfile specifies an app-specific environment. New in Bundler 0.9, you may store the .gem files and unpacked gems in a central location, as well as in the application itself.

Because Bundler ships with a dependency resolver, you do not need to specify named environments and switch between them. Instead, the environments are virtual, based on the gems you specify as application dependencies in the Gemfile, and you still get all the benefits of a shared, system-wide cache.

SafeBuffers and Rails 3.0

As you may have read, Rails adds XSS protection by default in Rails 3. This means that you no longer have to manually escape user input with the h helper, because Rails will automatically escape it for you.

However, it’s not as simple as all that. Consider the following:

Hello <strong>friends</strong>!
 
<%= tag(:p, some_text) %>
<%= some_text %>

In the above example, we have a few different scenarios involving HTML tags. First off, Rails should not escape the strong tag surrounding “friends”, because it is unambiguously not user input. Second, Rails should escape some_text in the <p> tag, but not the <p> tag itself. Finally, the some_text in the final tag should be escaped.

If some_text is <script>evil_js</script>, the above should output:

Hello <strong>friends</strong>!
 
<p>&lt;script&gt;evil_js&lt;/script&gt;</p>
&lt;script&gt;evil_js&lt;/script&gt;

In order to make this happen, we have introduced a new pervasive concept called html_safe into Rails applications. If a String is html_safe (which Rails determines by calling html_safe? on the String), ERB may insert it unaltered into the output. If it is not safe, ERB must first escape it before inserting it into the output.

def tag(name, options = nil, open = false, escape = true)
  "<#{name}#{tag_options(options, escape) if options}#{open ? ">" : " />"}".html_safe
end

Here, Rails creates the tag, telling tag_options to escape the contents, and then marks the entire body as safe. As a result, the <p> and </p> will emerge unaltered, while Rails will escape the user-supplied content.

The first implementation of this, in Koz’s rails-xss plugin, accomplished the above requirements by adding a new flag to all Strings. Rails, or Rails applications, could mark any String as safe, and Rails overrode + and << to mark the resulting String appropriately based on the input Strings.

However, during my last performance pass of Rails, I noticed that overriding every String concatenation resulted in quite a bit of performance overhead. Worse, the performance overhead was linear with the number of <%= %> in a template, so larger templates didn’t absorb the cost (as they would if the problem was once-per-template).

Thinking about the problem more, I realized (and confirmed with Koz, Jeremy, and Evan Phoenix of Rubinius), that we could implement roughly the same feature-set in a more performant way with a smaller API impact on Ruby. Because the problem itself is reasonably complex, I won’t go into a lot of detail about the old implementation, but will explain how you should use the XSS protection with the new implementation. If you already used Koz’s plugin or are working with the prereleases of Rails, you’ll notice that today’s commit changes very little.

SafeBuffer

In Rails 3, the ERB buffer is an instance of ActiveSupport::SafeBuffer. SafeBuffer inherits from String, overriding +, concat and << so that:

  • If the other String is safe (another SafeBuffer), the buffer concatenates it directly
  • If the other String is unsafe (a plain String), the buffer escapes it first, then concatenates it

Calling html_safe on a plain String returns a SafeBuffer wrapper. Because SafeBuffer inherits from String, Ruby creates this wrapper extremely efficiently (just sharing the internal char * storage).

As a result of this implementation, I was starting to see a lot of the following idiom in the codebase:

buffer << other_string.html_safe

Here, Rails is creating a new SafeBuffer for the other_string, then passing it to the << method of the original SafeBuffer, which then checks to see if it is safe. For cases like this, I created a new safe_concat method on the buffer which uses the original, native concat method, skipping both the need to create a new SafeBuffer and the need to check it.

Similarly, concat and safe_concat in ActionView proxy to the concat and safe_concat on the buffer itself, so you can use safe_concat in a helper if you have some HTML you want to concatenate to the buffer with no checks and without escaping.

ERB uses safe_concat internally on the parts of the template outside of <% %> tags, which means that with the changes I pushed today, the XSS protection code adds no performance impact to those cases (basically, all of the plain text in your templates).

Finally, ERB can now detect the raw helper at compile time, so if you do something like <%= raw some_stuff %>, ERB will use safe_concat internally, skipping the runtime creation of a SafeBuffer and checks for html_safety.

Summary

In summary, the XSS protection has the following characteristics:

  • If a plain String is passed into a <%= %>, Rails always escapes it
  • If a SafeBuffer is passed into a <%= %>, Rails does not escape it. To get a SafeBuffer from a String, call html_safe on it. The XSS system has a very small performance impact on this case, limited to a guard calling the html_safe? method
  • If you use the raw helper in a <%= %>, Rails detects it at compile-time of the template, resulting in zero performance impact from the XSS system on that concatenation
  • Rails does not escape any part of a template that is not in an ERB tag. Because Rails handles this at template compile-time, this results in zero performance impact from the XSS system on these concatenations

In comparison, the initial implementation of XSS impacted each concatenation or + of String, had impact even if the app used the raw helper, and even on plain Strings in templates.

That said, I want to extend personal thanks to Koz for getting the first draft out the door. It worked, demonstrated the concept, and let the community test it out. All in all, an excellent first pass.

Corporations Cannot Have Natural Rights… Duh

When I first read through the Citizens United decision that essentially made it illegal to “discriminate” against corporate political speech, I found myself very torn. On the one hand, I’m an extremely avid supporter of free speech rights, and Kennedy wrote the majority decision to tug at my heartstrings:

Premised on mistrust of governmental power, the First Amendment stands against attempts to disfavor certain subjects or viewpoints or to distinguish among different speakers, which may be a means to control content.

On the other hand, something felt wrong.

In the days that followed, I realized the the court had pulled a bait-and-switch, finding a way to split defenders of free speech, preventing a consistent response to the ruling, and resulting in a fair bit of liberal support for the ruling.

If you read the above quote closely, you can see the lie hiding in plain sight. The court, without drawing much notice, calls corporations “speakers” in the context of the First Amendment.

The problem is simple. The Bill of Rights exists to protect, for the long term, a set of natural rights. If you don’t believe me, read the writings of the anti-federalists, the guys who argued against the constitution and successfully agitated for the addition of the Bill of Rights in the first place:

That insatiable thirst for unconditional controul over our fellow-creatures, and the facility of sounds to convey essentially different ideas, produced the first Bill of Rights ever prefixed to a Frame of Government. The people, although fully sensible that they reserved every tittle of power they did not expressly grant away, yet afraid that the words made use of, to express those rights so granted might convey more than they originally intended, they chose at the same moment to express in different language those rights which the agreement did not include, and which they never designed to part with, endeavoring thereby to prevent any cause for future altercation and the intrusion into society of that doctrine of tacit implication which has been the favorite theme of every tyrant from the origin of all governments to the present day.

The language is a big crufty, but the meaning is clear. Without government, people have certain rights. They choose to surrender some of them to the government. The Constitution outlines the powers they have surrendered, and the Bill of Rights outlines specific cases where those powers may not tread.

However, corporations do not exist outside of government. They are not “creatures”. Instead, corporations are creations of the government, and therefore cannot reserve any rights via a bill of rights.

Looking at the quote from the ruling again:

Premised on mistrust of governmental power, the First Amendment stands against attempts to disfavor certain subjects or viewpoints or to distinguish among different speakers, which may be a means to control content.

The ruling fundamentally subverts that actual purpose of the Bill of Rights, using language that appeals to defenders of free speech to apply to entities that, by their very definition, the First Amendment cannot cover.

In short, the Bill of Rights protects the natural rights of people who have already surrendered a fair bit of their autonomy to the federal government. Corporations, by definition, cannot have natural rights, and since they don’t exist outside of government, cannot have surrendered any autonomy to the government. Therefore, the Bill of Rights, and the First Amendment, cannot apply to them. QED.

ActiveModel: Make Any Ruby Object Feel Like ActiveRecord

Rails 2.3 has a ton of really nice functionality locked up in monolithic components. I’ve posted quite a bit about how we’ve opened up a lot of that functionality in ActionPack, making it easier to reuse the router, dispatcher, and individual parts of ActionController. ActiveModel is another way we’ve exposed useful functionality to you in Rails 3.

Before I Begin, The ActiveModel API

Before I begin, there are two major elements to ActiveModel. The first is the ActiveModel API, the interface that models must adhere to in order to gain compatibility with ActionPack’s helpers. I’ll be talking more about that soon, but for now, the important thing about the ActiveModel API is that your models can become ActiveModel compliant without using a single line of Rails code.

In order to help you ensure that your models are compliant, ActiveModel comes with a module called ActiveModel::Lint that you can include into your test cases to test compliance with the API:

class LintTest < ActiveModel::TestCase
  include ActiveModel::Lint::Tests
 
  class CompliantModel
    extend ActiveModel::Naming
 
    def to_model
      self
    end
 
    def valid?()      true end
    def new_record?() true end
    def destroyed?()  true end
 
    def errors
      obj = Object.new
      def obj.[](key)         [] end
      def obj.full_messages() [] end
      obj
    end
  end
 
  def setup
    @model = CompliantModel.new
  end
end

The ActiveModel::Lint::Tests provide a series of tests that are run against the @model, testing for compliance.

ActiveModel Modules

The second interesting part of ActiveModel is a series of modules provided by ActiveModel that you can use to implement common model functionality on your own Ruby objects. These modules were extracted from ActiveRecord, and are now included in ActiveRecord.

Because we’re dogfooding these modules, you can be assured that APIs you bring in to your models will remain consistent with ActiveRecord, and that they’ll continue to be maintained in future releases of Rails.

The ActiveModel comes with internationalization baked in, providing an avenue for much better community sharing around translating error messages and the like.

The Validations System

This was perhaps the most frustrating coupling in ActiveRecord, because it meant that people writing libraries for, say, CouchDB had to choose between painstakingly copying the API over, allowing inconsistencies to creep in, or just inventing a whole new API.

Validations have a few different elements.

First, declaring the validations themselves. You’ve seen the usage before in ActiveRecord:

class Person < ActiveRecord::Base
  validates_presence_of :first_name, :last_name
end

To do the same thing for a plain old Ruby object, simply do the following:

class Person
  include ActiveModel::Validations
 
  validates_presence_of :first_name, :last_name
 
  attr_accessor :first_name, :last_name
  def initialize(first_name, last_name)
    @first_name, @last_name = first_name, last_name
  end
end

The validations system calls read_attribute_for_validation to get the attribute, but by default, it aliases that method to send, which supports the standard Ruby attribute system of attr_accessor.

To use a more custom attribute lookup, you can do:

class Person
  include ActiveModel::Validations
 
  validates_presence_of :first_name, :last_name
 
  def initialize(attributes = {})
    @attributes = attributes
  end
 
  def read_attribute_for_validation(key)
    @attributes[key]
  end
end

Let’s look at what a validator actually is. First of all, the validates_presence_of method:

def validates_presence_of(*attr_names)
  validates_with PresenceValidator, _merge_attributes(attr_names)
end

You can see that validates_presence_of is using the more primitive validates_with, passing it the validator class, merging in {:attributes => attribute_names} into the options passed to the validator. Next, the validator itself:

class PresenceValidator < EachValidator
  def validate(record)
    record.errors.add_on_blank(attributes, options[:message])
  end
end

The EachValidator that it inherits from validates each attribute with the validate method. In this case, it adds the error message to the record, only if the attribute is blank.

The add_on_blank method does add(attribute, :blank, :default => custom_message) if value.blank? (among other things), which is adding the localized :blank message to the object. If you take a look at the built-in locale/en.yml looks like:

en:
  errors:
    # The default format use in full error messages.
    format: "{{attribute}} {{message}}"
 
    # The values :model, :attribute and :value are always available for interpolation
    # The value :count is available when applicable. Can be used for pluralization.
    messages:
      inclusion: "is not included in the list"
      exclusion: "is reserved"
      invalid: "is invalid"
      confirmation: "doesn't match confirmation"
      accepted: "must be accepted"
      empty: "can't be empty"
      blank: "can't be blank"
      too_long: "is too long (maximum is {{count}} characters)"
      too_short: "is too short (minimum is {{count}} characters)"
      wrong_length: "is the wrong length (should be {{count}} characters)"
      not_a_number: "is not a number"
      greater_than: "must be greater than {{count}}"
      greater_than_or_equal_to: "must be greater than or equal to {{count}}"
      equal_to: "must be equal to {{count}}"
      less_than: "must be less than {{count}}"
      less_than_or_equal_to: "must be less than or equal to {{count}}"
      odd: "must be odd"
      even: "must be even"

As a result, the error message will read first_name can't be blank.

The Error object is also a part of ActiveModel.

Serialization

ActiveRecord also comes with default serialization for JSON and XML, allowing you to do things like: @person.to_json(:except => :comment).

The main important part of the serialization support is adding general support for specifying the attributes to include across all serializers. That means that you can do @person.to_xml(:except => :comment) as well.

To add serialization support to your own model, you will need to include the serialization module and implement attributes. Check it out:

class Person
  include ActiveModel::Serialization
 
  attr_accessor :attributes
  def initialize(attributes)
    @attributes = attributes
  end
end
 
p = Person.new(:first_name => "Yukihiro", :last_name => "Matsumoto")
p.to_json #=> %|{"first_name": "Yukihiro", "last_name": "Matsumoto"}|
p.to_json(:only => :first_name) #=> %|{"first_name": "Yukihiro"}|

You can also pass in a :methods option to specify methods to call for certain attributes that are determined dynamically.

Here's the Person model with validations and serialization:

class Person
  include ActiveModel::Validations
  include ActiveModel::Serialization
 
  validates_presence_of :first_name, :last_name
 
  attr_accessor :attributes
  def initialize(attributes = {})
    @attributes = attributes
  end
 
  def read_attribute_for_validation(key)
    @attributes[key]
  end
end

Others

Those are just two of the modules available in ActiveModel. Some others include:

  • AttributeMethods: Makes it easy to add attributes that are set like table_name :foo
  • Callbacks: ActiveRecord-style lifecycle callbacks.
  • Dirty: Support for dirty tracking
  • Naming: Default implementations of model.model_name, which are used by ActionPack (for instance, when you do render :partial => model
  • Observing: ActiveRecord-style observers
  • StateMachine: A simple state-machine implementation for models
  • Translation: The core translation support

This mostly reflects the first step of ActiveRecord extractions done by Josh Peek for his Google Summer of Code project last summer. Over time, I expect to see more extractions from ActiveRecord and more abstractions built up around ActiveModel.

I also expect to see a community building up around things like adding new validators, translations, serializers and more, especially now that they can be reused not only in ActiveRecord, but in MongoMapper, Cassandra Object, and other ORMs that leverage ActiveModel's built-in modules.

The Maximal Usage Doctrine for Open Source

I’ve worked on a number of open source projects over the past several years (the most prominent being Merb, Ruby on Rails and jQuery) and have begun to form some thoughts about the usage (aka adoption) of open source projects and the practical effects of license styles.

The Playing Field

There are essentially two kinds of licenses popularly used in open source[1].

The type I’ve worked with most extensively in the BSD or MIT-style license. This license allows unlimited usage, modification, distribution, and commercialization of the source code, with two caveats. First, the copyright notice must be distributed with the source code. Second, to the extent legally enforceable, if you use my MIT-licensed source code, you may not sue me for the effects of using the source.

The other major license type is called copyleft, and essentially leverages copyright law to insist that any modifications to the original source must themselves be made public. The practical effect of this license could not be effected without government-enforced copyright laws.

The most popular copyleft license is GPL, which is used by Linux, and which also has viral characteristics. In short, if you link code covered by the GPL into your own code, you are required to open your code as well (if you distribute it). The GPL is not very clear about what this means for more dynamic languages (is requiring a Ruby file “linking”?).

A less viral version of the GPL, the LGPL, requires that any modifications you make to the software itself be released, but drops the “linking” requirement.

Because of the uncertainty surrounding the viral characteristics of the GPL, legal departments in big corporations are very hostile to the GPL license. Software covered by the LGPL license is more likely to get approval by corporate legal, and MIT/BSD licenses are the most well-liked (probably because they don’t confer any obligations on the corporation).

What I Want When I Write Open Source

When I work on a serious open source project, like Ruby on Rails or jQuery, I have relatively simple desires.

One. I want as many people as possible to use the software I am working on. This is probably a selfish, egoist desire, but it may also be a desire to see useful software actually used.

Two. I want some subset of the many users to be willing to report bugs, and for the varied situations the code is used in to help improve the quality of the software as a result. This is essentially “given enough eyeballs, all bugs are shallow”.

Three. I want people who are interested in improving the software to submit patches. Specifically, I want to receive patches from people who have thought through the problem the bug is trying to fix, and want to receive fewer patches from people who have hacked together a solution just to get it to work. In essence, a high signal-to-noise ratio of patches is more important than a high volume of patches.

Four. I want people who are interested in improving the software long term to become long-term contributors and eventually committers.

Of these concerns, numbers two and three are the highest priorities. I want to expose the code to as much real-world pressure as possible, and I want as many high-quality patches to fix those bugs as possible. If I had to choose between those two, I would pick number two: exposing my code to real-world stress is the best way to rapidly ferret out bugs and incorrect design assumptions that I know of.

Meeting Those Requirements

Starting out with the easiest, my first desire, to have my software used as much as possible, is most easily satisfied by an extremely liberal usage policy. Adding restrictions on the use of software I write reduces its adoption almost by definition.

Much more importantly, the same can be said about exposing code to real world stresses. By far the most important way to achieve this goal is to make it as easy as possible for as many people as possible to use the code.

If only 1% of all proprietary users of the source ever report bugs, that’s 1% of potentially thousands of users, as opposed to 100% of the zero proprietary users who were able to use the software under a more restrictive usage scheme. In practice, this number is much more than 1%, as proprietary users of software experience and report bugs just like open source users do.

The only real counter-argument to this is that by forcing users to contribute, some number of proprietary users will be forced to become open source users, and their contributions will outweigh the smaller contributions of proprietary users. In practice, proprietary users choose proprietary solutions instead when they are forced to choose between restrictive open source usage schemes and other proprietary software.

There is also much to be said for exposing open source tools into proprietary environments.

There is also much to be said for introducing proprietary developers to the open source ecosystem. Proprietary developers have access to things like paid time, access to obscure usage scenarios, and access to markets with low open source penetration that the open source community lacks.

The next major desire I have while working on open source is a steady stream of high-quality patches. This should be advantage copyleft, because all users of the software are forced to contribute back. However, since copyleft licenses are not used in proprietary environments anyway, the patches to open source projects from those environments under more permissive licenses are much more numerous. Again, even if only a few percent of proprietary users contribute back to the project, that is significantly more contributions than the 100% of zero proprietary users.

Also importantly, the patches are contributed by much more dedicated users of the software, instead of being force-contributions. I have never heard a team member on an open source project say that inadequate patches are received by permissive-license software, and that this problem would be solved by going to a more restrictive model.

Finally, I can only speak for jQuery and Rails (and other smaller open source projects I work on), but a large number of new long-term contributors became involved while working on corporate projects, where a permissive license made the decision to use the software in the first place feasible.

Lower Barrier to Entry Helps Meet Open Source Goals

Regardless of how much of the above argument you agree with, it is clear that copyleft licenses intentionally impose a higher barrier to entry for usage than more permissive licenses.

For projects that use more permissive licenses, the fact that many proprietary users of their software do not contribute back changes is a feature, not a bug.

That’s because we don’t focus on all of the people who use our software without contributing back. Instead, we focus on all the users, bug reports, patches, and long term contributors we have gained by keeping the barrier to entry as low as possible.

In the end, the world of ubiquitous open source is here. And we made it without having to resort to coercion or the formation of an entirely self-contained set of software. We made it by building great software and convincing real users that it was better than the alternative.

Postscript: Linux

Linux is a peculiar example because its license has not impeded its usage much. In part, that is because most users of Linux do not make changes to it, and the linking characteristics of the GPL license rarely come into play.

In cases like this, I would argue that copyleft licenses are close enough to usage maximization to get most of the benefits of more permissive licenses. However, it’s not clear to me what benefits the copyleft licenses provide in those (relatively rare) cases.

[1] There are other kinds of licenses, like the Affero license, which is even more restrictive than the GPL license. I would classify Affero as copyleft.

Archives

Categories

Meta