Yehuda Katz is a member of the Ember.js, Ruby on Rails and jQuery Core Teams; his 9-to-5 home is at the startup he founded, Tilde Inc.. There he works on Skylight, the smart profiler for Rails, and does Ember.js consulting. He is best known for his open source work, which also includes Thor and Handlebars. He travels the world doing open source evangelism and web standards work.

Simplifying Rails Block Helpers (With a Side of Rubinius)

We all know that <%= string %> emits a String in ERB. And <% string %> runs Ruby code, but does not emit a String. When starting working with Rails, you almost expect the syntax for block helpers to be:

<%= content_tag(:div) do %>
  The content
<% end %>

Why doesn’t it work that way?

It has to do with how the ERB parser works, looking at each line individually. When it sees <% %>, it evaluates the code as a line of Ruby. When it sees <%= %>, it evaluates the inside of the ERB tag, and calls to_s on it.

This:

<% form_for(@object) do %>
Stuff
<% end %>

gets effectively converted to:

form_for(@object) do
_buf << ("Stuff").to_s
end

On the other hand, this:

<%= form_for(@object) do %>
Stuff
<% end %>

gets converted to:

_buf << (form_for(@object) do).to_s
_buf << ("Stuff").to_s
end

which isn’t valid Ruby. So we use the first approach, and then let the helper itself, rather than ERB, be responsible for concatenating to the buffer. Sadly, it leads to significantly more complex helpers.

Let’s take a look at the implementation of content_tag.

def content_tag(name, content_or_options_with_block = nil, options = nil, escape = true, &block)
  if block_given?
    options = content_or_options_with_block if content_or_options_with_block.is_a?(Hash)
    content_tag = content_tag_string(name, capture(&block), options, escape)
 
    if block_called_from_erb?(block)
      concat(content_tag)
    else
      content_tag
    end
  else
    content_tag_string(name, content_or_options_with_block, options, escape)
  end
end

The important chunk here is the middle, inside of the if block_given? section. The first few lines just get the actual contents, using the capture helper to pull out the contents of the block. But then you get this:

if block_called_from_erb?(block)
  concat(content_tag)
else
  content_tag
end

This is actually a requirement for writing a block helper of any kind in Rails. First, Rails checks to see if the block is being called from ERB. If so, it takes care of concatenating to the buffer. Otherwise, the caller simply wants a String back, so it returns it.

Worse, here’s the implementation of block_called_from_erb?:

BLOCK_CALLED_FROM_ERB = 'defined? __in_erb_template'
 
# Check whether we're called from an erb template.
# We'd return a string in any other case, but erb <%= ... %>
# can't take an <% end %> later on, so we have to use <% ... %>
# and implicitly concat.
def block_called_from_erb?(block)
  block && eval(BLOCK_CALLED_FROM_ERB, block)
end

So every time you use a block helper in Rails, or use a helper which uses a block helper, Rails is forced to eval into the block to determine what the context is.

In Merb, we solved this problem by using this syntax:

<%= form_for(@object) do %>
Stuff
<% end =%>

And while everyone agrees that the opening <%= is a reasonable change, the closing =%> is a bit grating. However, it allows us to compile the above code into:

_buf << (form_for(@object) do
_buf << ("Stuff").to_s
end).to_s

That’s because we tag the end with a special ERB tag that allows us to attach a ).to_s to the end. We use Erubis, which lets us control the compilation process more finely, to hook into this process.

Rails 3 will use Erubis regardless of this problem to implement on-by-default XSS protection, but I needed a solution that didn’t require the closing =%> (ideally).

Evan (lead on Rubinius) hit upon a rather ingenious idea: use Ruby operator precedence to get around the need to know where the end was. Effectively, compile into the following:

_buf << capture_obj << form_for(@object) do
_buf << ("Stuff").to_s
end

where capture_obj is:

class CaptureObject
  def <<(obj)
    @object = obj
    self
  end
 
  def to_str
    @object.to_s
  end
 
  def to_s
    @object.to_s
  end
end

Unfortunately, with one hand Ruby operator precedence giveth, and with one hand it taketh away. In order to test this, I tried using a helper that returned an object, rather than a String (valid in ERB). In ERB, this would call to_s on the object. When I tried to run this code with the CaptureObject, I got:

template template:1:in `<<': can't convert Object into String (TypeError)
   from template template:1:in `template'
   from helper_spike.rb:48

Evan and I were both a bit baffled by this (although it retrospect we probably shouldn’t have been), and we hit on the idea to try running the code through Rubinius and look at its backtrace:

An exception occurred running helper_spike.rb
    Coercion error: #<Object:0x60a>.to_str => String failed:
(No method 'to_str' on an instance of Object.) (TypeError)
 
Backtrace:
                       Type.coerce_to at kernel/common/type.rb:22
           Kernel(String)#StringValue at kernel/common/kernel.rb:82
                            String#<< at kernel/common/string.rb:93
                   MyContext#template at template template:1
                      main.__script__ at helper_spike.rb:48

By looking at Rubinius’ backtrace, we quickly realized that the order of operations was wrong, and to_str was getting called on the return value from the helper, rather than the CaptureObject. As I tweeted immediately thereafter, the information available in Rubinius’ backtrace is just phenomenal, exposing enough information to really see what’s going on. Because the internals of Rubinius are written in Ruby, the Ruby backtrace goes all the way through to the Type.coerce_to method.

After realizing that, we changed the implementation of CaptureObject to take the buffer in its initializer, and have it handle concatenating to the buffer. The compiled code now looks like:

capture_obj << form_for(@object) do
_buf << ("Stuff").to_s
end

and the CaptureObject looks like:

class CaptureObject
  def initialize(buf)
    @buf = buf
  end
 
  def <<(obj)
    @buf << obj.to_s
  end
end

Now, Ruby’s operator precedence will bind the do to the form_for, and the return value of form_for will be to_s‘ed and concatenated to the buffer.

And the best thing is the implementation of content_tag once that’s done:

def content_tag(name, content = nil, options = nil, escape = true, &block)
  if block_given?
    options = content if content.is_a?(Hash)
    content = capture(&block)
  end
  content_tag_string(name, content, options, escape)
end

We can simply return a String and ERB handles the concatenation work. That’s the important part: helper writers should be able to think of block helpers the same way they think about traditional helpers. Somewhat less importantly, we’ll be able to eliminate evaling into untold numbers of blocks at runtime.

This was only an experiment, and the specific details still need to be worked out (how do we do this without breaking untold numbers of existing applications), I’m very happy with this solution, which provides the simplicity and performance enhancement of the Merb solution without the ugly =%>.

15 Responses to “Simplifying Rails Block Helpers (With a Side of Rubinius)”

Hey,

these kind of articles are really eye opening / insightful and its so great you put so much effort in refactoring out the ugly parts of rails. Although I think I understood what you’ve been doing I would be still interested to see what the implementation of the capture method looks like. I suppose it changed as well as you would not rely on block_called_from_erb?(block) anymore right ?

kind regards, John

This is what capture looked like in my spike:

  def capture
    old_buf, @_buf = @_buf, ""
    yield
    @_buf
  ensure
    @_buf = old_buf
  end

In Ralls it would likely look like:

def capture(*args, &block)
  with_output_buffer { return block.call(*args) }
end

Hmm,

I should get more familiar with the rails internals or ERB as I don’t understand exactly whats happening. Like the ERB gets processed and a block (helper) is encountered. Now this helper gets parsed / evaluated by ERB. This way I would guess you’re patching ERB from the rails world.

I’d love to understand how rails really works but it seems just such a large codebase that I don’t know where to start or how not to get lost. Do you have any tips on how to get to know rails from inside out? I can clearly see the benefits of this patch but it is unsatisfying for me that I don’t fully understand what’s going on.

John

What a great post! I was refactoring a gem the other week, that was forced to place concat all throughout in order to output the contents to a block. Not only did it make code harder to read, but testing was a nightmare.

Being able to do <%= on a method with a block would be a huge bonus! Keep up the great work!

… great work, my websites will feel less dirty now ;).

I’m loving all these posts about Ruby and how you’re making it better. Thanks for your work on Rails and thanks for your insights into Ruby code like this one.

Freaking brilliant! I’ve always wanted this syntax – thanks for figuring out how to pull it off and documenting the journey.

Love it, you guys rock! You took something that always bugged me, fixed it and made it awesome. <3

Woah! That is simply astounding. And simple.

Also, seeing that backtrace in Rubinius made me feel all warm and fuzzy. If only we had that in MRI!!

Oh, and I was wondering whether y’all would be using Erubis, considering it beats the pants off of ERB and all ;)

This is pretty sweet! As others have said, I’m always for cleaning up the dirty insides even when the outside looks great. I’m really looking forward to Rails 3.

Katz, I just want to send a big “Thanks!” for all of the posts you’ve been writing lately on Rails internals. I’m learning tons from your examples; you must be going for Rails blogger of the year award or something, and you’re doing a damn fine job. Thanks again and please keep up the great writing!

Thanks for blog post – Rubinius looks really useful for things like this

“helper writers should be able to think of block helpers the same way they think about traditional helpers”

completely agree – so I’ve written a block helpers gem you might find interesting at http://github.com/markevans/block_helpers
( blog post at http://blog.new-bamboo.co.uk/2009/8/14/block-helpers )

I’d love to see that kind of thing in Rails

One word; awesome!

Why couldn’t erb have, say, this method:

def _erb_write_(stuff); _buf << stuff.to_s end

and then replace "” with “_erb_write_ foo”?

Then you only need to know about one end of the block. the “end” part doesn’t need to be checked.

Could it break some other obscure expressions? Is it perhaps worth it?

Yes I know _buf is local and might need to become an instance var or similar.

Also the function to perform that operation could also be exposde to template code to be used inside as a consistent method for adding to the output inside a larger block of code.

Sorry about the late reply :)

Leave a Reply

Archives

Categories

Meta