Rack as a Transformative Figure

I had occasion to think about Rack a few times today. First of all, I was writing some proposals to a few conferences about Rails, which always gets me thinking about architecture. In this case, it got me thinking about how Rack is influencing Rails' architecture moving forward. Second, a friend told me he was considering moving from an archaic web language to Python (specifically Django). I got to thinking about how the future of Ruby is intertwined with Rack, and why that might not be the case with Python and WSGI, unfortunately.

Like Ruby, Python has a single dominant web framework in Django. According to Google Trends, Django gets about eight times as many programming-related searches as the next-most-popular framework, Turbogears. And TurboGears is 1.5 to 2x the size of Pylons and CherryPy, other frameworks. Naturally, Rails dominates the Ruby space more thoroughly (in the last 12 months, Rails beat the next-most-popular framework, Merb, by more than 60 times), but the effect is roughly the same. Many more people contribute to the Django and Rails ecosystem than the Turbogears and Merb ecosystems.

Fortunately for Ruby, however, Rails (our dominant framework) has thoroughly embraced Rack, an abstraction layer and specification for the code between a web framework and a web server. This embrace of Rack has brought much-needed oxygen to the specification, and has allowed Merb, Rails, Sinatra, and others to cooperate in a shared space. In the next few months, Merb and Rails will be making their routers a shared Rack component, and the same is true for a number of smaller elements, like parameter parsing. In part, this is possible because unlike WSGI, Rack is a specification and a library. In the WSGI universe, the WebOb library provides similar functionality, but is not as widely shared.

In particular, while Django itself can run on WSGI servers, Django does not use WebOb, nor is its middleware WSGI-compliant. Let's look at a specific example, a piece of middleware that wraps all requests in Django in a transaction:

from django.db import transaction

class TransactionMiddleware(object):
    """
    Transaction middleware. If this is enabled, each view function will be run
    with commit_on_response activated - that way a save() doesn't do a direct
    commit, the commit is done when a successful response is created. If an
    exception happens, the database is rolled back.
    """
    def process_request(self, request):
        """Enters transaction management"""
        transaction.enter_transaction_management()
        transaction.managed(True)

    def process_exception(self, request, exception):
        """Rolls back the database and leaves transaction management"""
        if transaction.is_dirty():
            transaction.rollback()
        transaction.leave_transaction_management()

    def process_response(self, request, response):
        """Commits and leaves transaction management."""
        if transaction.is_managed():
            if transaction.is_dirty():
                transaction.commit()
            transaction.leave_transaction_management()
        return response

As you can see, Django allows multiple methods in its middleware (process_request, process_response, process_exception, and process_view). As with rack, WSGI middleware uses a single method to wrap the entire request. Here's how the above Django middleware would look in Rack:

class TransactionMiddleware
  def initialize(app) @app = app end
  
  def call(env)
    transaction.enter_transaction_management
    tranaction.managed(true)  
    begin
      @app.call(env)
    rescue
      transaction.rollback if transaction.dirty?
      transaction.leave_transaction_management
    end
    if transaction.managed?
      transaction.commit if transaction.dirty?
      transaction.leave_transaction_management
    end
  end
end

Leaving aside the differences in language, the Django middleware is incompatible with WSGI for no discernable reason. This means that Django middleware cannot be shared with the rest of the (admittedly smaller) Python web ecosystem, which makes it harder, for instance, to use Django's ORM separately from Django (because you'd have to write your own transaction-wrapping middleware).

The big win of WSGI and Rack is that like Unix pipes, they make it easy to mix and match web framework elements to develop something that works for you. It makes it easier to develop entire frameworks, like CloudKit, that can be used as simple plugins to larger frameworks like Rails. Rails' embrace of Rack, in the long term, is going to significantly open up experimentation in the community to non-Rails solutions that can be used WITH RAILS while they are being ironed out. In fact, being able to trivially run a Sinatra app inside a Rails app is an important design goal for Rails moving forward.

When you look inside the Rails middleware folder, you see things like this in the Rails Failsafe middleware:

def call(env)
  @app.call(env)
rescue Exception => exception
  # Reraise exception in test environment
  if env["rack.test"]
    raise exception
  else
    failsafe_response(exception)
  end
end

Instead of assuming something specific about Rails test mode, this middleware is allowing the community to come together around specifications that all Ruby frameworks can use together (in this case, a generic "test mode"). And this middleware can be used in Sinatra, Merb, or CloudKit by simply requiring "action_dispatch/middleware/failsafe" (as of Rails3) and then adding ActionDispatch::FailSafe as a middleware. No other Rails assumptions are made.

The bottom line is that this is going to be very powerful moving forward, and it's too bad that Django's insistence on a go-it-your-alone approach is damaging the Python community's attempts to do the same.

On a side note, I want to be clear that I am aware that Django can run inside of a WSGI server, and can use WSGI middleware. However, Django middleware cannot be used by WSGI, and a recent post on the topic, which advocated the use of WSGI middleware in Django apps, had to be sure to include "I'm not trying to stir up any controversy, I'm not saying we should stop making Django middleware or anything like that". I, for one, think that's a shame.