Simplifying Rails Block Helpers (With a Side of Rubinius)

August 31st, 2009

We all know that <%= string %> emits a String in ERB. And <% string %> runs Ruby code, but does not emit a String. When starting working with Rails, you almost expect the syntax for block helpers to be:

<%= content_tag(:div) do %>
  The content
<% end %>

Why doesn’t it work that way?

It has to do with how the ERB parser works, looking at each line individually. When it sees <% %>, it evaluates the code as a line of Ruby. When it sees <%= %>, it evaluates the inside of the ERB tag, and calls to_s on it.

This:

<% form_for(@object) do %>
Stuff
<% end %>

gets effectively converted to:

form_for(@object) do
_buf << ("Stuff").to_s
end

On the other hand, this:

<%= form_for(@object) do %>
Stuff
<% end %>

gets converted to:

_buf << (form_for(@object) do).to_s
_buf << ("Stuff").to_s
end

which isn’t valid Ruby. So we use the first approach, and then let the helper itself, rather than ERB, be responsible for concatenating to the buffer. Sadly, it leads to significantly more complex helpers.

Let’s take a look at the implementation of content_tag.

def content_tag(name, content_or_options_with_block = nil, options = nil, escape = true, &block)
  if block_given?
    options = content_or_options_with_block if content_or_options_with_block.is_a?(Hash)
    content_tag = content_tag_string(name, capture(&block), options, escape)
 
    if block_called_from_erb?(block)
      concat(content_tag)
    else
      content_tag
    end
  else
    content_tag_string(name, content_or_options_with_block, options, escape)
  end
end

The important chunk here is the middle, inside of the if block_given? section. The first few lines just get the actual contents, using the capture helper to pull out the contents of the block. But then you get this:

if block_called_from_erb?(block)
  concat(content_tag)
else
  content_tag
end

This is actually a requirement for writing a block helper of any kind in Rails. First, Rails checks to see if the block is being called from ERB. If so, it takes care of concatenating to the buffer. Otherwise, the caller simply wants a String back, so it returns it.

Worse, here’s the implementation of block_called_from_erb?:

BLOCK_CALLED_FROM_ERB = 'defined? __in_erb_template'
 
# Check whether we're called from an erb template.
# We'd return a string in any other case, but erb <%= ... %>
# can't take an <% end %> later on, so we have to use <% ... %>
# and implicitly concat.
def block_called_from_erb?(block)
  block && eval(BLOCK_CALLED_FROM_ERB, block)
end

So every time you use a block helper in Rails, or use a helper which uses a block helper, Rails is forced to eval into the block to determine what the context is.

In Merb, we solved this problem by using this syntax:

<%= form_for(@object) do %>
Stuff
<% end =%>

And while everyone agrees that the opening <%= is a reasonable change, the closing =%> is a bit grating. However, it allows us to compile the above code into:

_buf << (form_for(@object) do
_buf << ("Stuff").to_s
end).to_s

That’s because we tag the end with a special ERB tag that allows us to attach a ).to_s to the end. We use Erubis, which lets us control the compilation process more finely, to hook into this process.

Rails 3 will use Erubis regardless of this problem to implement on-by-default XSS protection, but I needed a solution that didn’t require the closing =%> (ideally).

Evan (lead on Rubinius) hit upon a rather ingenious idea: use Ruby operator precedence to get around the need to know where the end was. Effectively, compile into the following:

_buf << capture_obj << form_for(@object) do
_buf << ("Stuff").to_s
end

where capture_obj is:

class CaptureObject
  def <<(obj)
    @object = obj
    self
  end
 
  def to_str
    @object.to_s
  end
 
  def to_s
    @object.to_s
  end
end

Unfortunately, with one hand Ruby operator precedence giveth, and with one hand it taketh away. In order to test this, I tried using a helper that returned an object, rather than a String (valid in ERB). In ERB, this would call to_s on the object. When I tried to run this code with the CaptureObject, I got:

template template:1:in `<<': can't convert Object into String (TypeError)
   from template template:1:in `template'
   from helper_spike.rb:48

Evan and I were both a bit baffled by this (although it retrospect we probably shouldn’t have been), and we hit on the idea to try running the code through Rubinius and look at its backtrace:

An exception occurred running helper_spike.rb
    Coercion error: #<Object:0x60a>.to_str => String failed:
(No method 'to_str' on an instance of Object.) (TypeError)
 
Backtrace:
                       Type.coerce_to at kernel/common/type.rb:22
           Kernel(String)#StringValue at kernel/common/kernel.rb:82
                            String#<< at kernel/common/string.rb:93
                   MyContext#template at template template:1
                      main.__script__ at helper_spike.rb:48

By looking at Rubinius’ backtrace, we quickly realized that the order of operations was wrong, and to_str was getting called on the return value from the helper, rather than the CaptureObject. As I tweeted immediately thereafter, the information available in Rubinius’ backtrace is just phenomenal, exposing enough information to really see what’s going on. Because the internals of Rubinius are written in Ruby, the Ruby backtrace goes all the way through to the Type.coerce_to method.

After realizing that, we changed the implementation of CaptureObject to take the buffer in its initializer, and have it handle concatenating to the buffer. The compiled code now looks like:

capture_obj << form_for(@object) do
_buf << ("Stuff").to_s
end

and the CaptureObject looks like:

class CaptureObject
  def initialize(buf)
    @buf = buf
  end
 
  def <<(obj)
    @buf << obj.to_s
  end
end

Now, Ruby’s operator precedence will bind the do to the form_for, and the return value of form_for will be to_s‘ed and concatenated to the buffer.

And the best thing is the implementation of content_tag once that’s done:

def content_tag(name, content = nil, options = nil, escape = true, &block)
  if block_given?
    options = content if content.is_a?(Hash)
    content = capture(&block)
  end
  content_tag_string(name, content, options, escape)
end

We can simply return a String and ERB handles the concatenation work. That’s the important part: helper writers should be able to think of block helpers the same way they think about traditional helpers. Somewhat less importantly, we’ll be able to eliminate evaling into untold numbers of blocks at runtime.

This was only an experiment, and the specific details still need to be worked out (how do we do this without breaking untold numbers of existing applications), I’m very happy with this solution, which provides the simplicity and performance enhancement of the Merb solution without the ugly =%>.

This entry was posted on Monday, August 31st, 2009 at 12:45 am and is filed under Rails 3, Ruby, Ruby on Rails. You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a response. Pinging is currently not allowed.

14 Responses to “Simplifying Rails Block Helpers (With a Side of Rubinius)”

hukl, Posted August 31, 2009, 2:15 am

Hey,

these kind of articles are really eye opening / insightful and its so great you put so much effort in refactoring out the ugly parts of rails. Although I think I understood what you’ve been doing I would be still interested to see what the implementation of the capture method looks like. I suppose it changed as well as you would not rely on block_called_from_erb?(block) anymore right ?

kind regards, John

Yehuda Katz, Posted August 31, 2009, 2:28 am

This is what capture looked like in my spike:

  def capture
    old_buf, @_buf = @_buf, ""
    yield
    @_buf
  ensure
    @_buf = old_buf
  end

In Ralls it would likely look like:

def capture(*args, &block)
  with_output_buffer { return block.call(*args) }
end

hukl, Posted August 31, 2009, 4:49 am

Hmm,

I should get more familiar with the rails internals or ERB as I don’t understand exactly whats happening. Like the ERB gets processed and a block (helper) is encountered. Now this helper gets parsed / evaluated by ERB. This way I would guess you’re patching ERB from the rails world.

I’d love to understand how rails really works but it seems just such a large codebase that I don’t know where to start or how not to get lost. Do you have any tips on how to get to know rails from inside out? I can clearly see the benefits of this patch but it is unsatisfying for me that I don’t fully understand what’s going on.

John

Kieran P, Posted August 31, 2009, 5:14 am

What a great post! I was refactoring a gem the other week, that was forced to place concat all throughout in order to output the contents to a block. Not only did it make code harder to read, but testing was a nightmare.

Being able to do <%= on a method with a block would be a huge bonus! Keep up the great work!

ippa, Posted August 31, 2009, 9:09 am

… great work, my websites will feel less dirty now ;).

Tiago Pinto, Posted August 31, 2009, 11:48 am

I’m loving all these posts about Ruby and how you’re making it better. Thanks for your work on Rails and thanks for your insights into Ruby code like this one.

Chris, Posted August 31, 2009, 4:02 pm

Freaking brilliant! I’ve always wanted this syntax – thanks for figuring out how to pull it off and documenting the journey.

Pete Nicholls, Posted August 31, 2009, 5:29 pm

Love it, you guys rock! You took something that always bugged me, fixed it and made it awesome. <3

Elliot Winkler, Posted August 31, 2009, 9:11 pm

Woah! That is simply astounding. And simple.

Also, seeing that backtrace in Rubinius made me feel all warm and fuzzy. If only we had that in MRI!!

Elliot Winkler, Posted August 31, 2009, 9:12 pm

Oh, and I was wondering whether y’all would be using Erubis, considering it beats the pants off of ERB and all ;)

Peter Wagenet, Posted September 1, 2009, 8:37 am

This is pretty sweet! As others have said, I’m always for cleaning up the dirty insides even when the outside looks great. I’m really looking forward to Rails 3.

Gabe Hollombe, Posted September 1, 2009, 7:47 pm

Katz, I just want to send a big “Thanks!” for all of the posts you’ve been writing lately on Rails internals. I’m learning tons from your examples; you must be going for Rails blogger of the year award or something, and you’re doing a damn fine job. Thanks again and please keep up the great writing!

Mark Evans, Posted September 2, 2009, 12:33 pm

Thanks for blog post – Rubinius looks really useful for things like this

“helper writers should be able to think of block helpers the same way they think about traditional helpers”

completely agree – so I’ve written a block helpers gem you might find interesting at http://github.com/markevans/block_helpers
( blog post at http://blog.new-bamboo.co.uk/2009/8/14/block-helpers )

I’d love to see that kind of thing in Rails

Eloy Duran, Posted September 8, 2009, 3:39 am

One word; awesome!