Yehuda Katz is a member of the Ember.js, Ruby on Rails and jQuery Core Teams; he spends his daytime hours at the startup he founded, Tilde Inc.. Yehuda is co-author of best-selling jQuery in Action and Rails 3 in Action. He spends most of his time hacking on open source—his main projects, like Thor, Handlebars and Janus—or traveling the world doing evangelism work. He can be found on Twitter as @wycats and on Github.

Metaprogramming in Ruby: It’s All About the Self

After writing my last post on Rails plugin idioms, I realized that Ruby metaprogramming, at its core, is actually quite simple.

It comes down to the fact that all Ruby code is executed code–there is no separate compile or runtime phase. In Ruby, every line of code is executed against a particular self. Consider the following five snippets:

class Person
  def self.species
    "Homo Sapien"
  end
end
 
class Person
  class << self
    def species
      "Homo Sapien"
    end
  end
end
 
class << Person
  def species
    "Homo Sapien"
  end
end
 
Person.instance_eval do
  def species
    "Homo Sapien"
  end
end
 
def Person.species
  "Homo Sapien"
end

All five of these snippets define a Person.species that returns Homo Sapien. Now consider another set of snippets:

class Person
  def name
    "Matz"
  end
end
 
Person.class_eval do
  def name
    "Matz"
  end
end

These snippets all define a method called name on the Person class. So Person.new.name will return “Matz”. For those familiar with Ruby, this isn’t news. When learning about metaprogramming, each of these snippets is presented in isolation: another mechanism for getting methods where they “belong”. In fact, however, there is a single unified reason that all of these snippets work the way they do.

First, it is important to understand how Ruby’s metaclass works. When you first learn Ruby, you learn about the concept of the class, and that each object in Ruby has one:

class Person
end
 
Person.class #=> Class
 
class Class
  def loud_name
    "#{name.upcase}!"
  end
end
 
Person.loud_name #=> "PERSON!"

Person is an instance of Class, so any methods added to Class are available on Person as well. What they don’t tell you, however, is that each object in Ruby also has its own metaclass, a Class that can have methods, but is only attached to the object itself.

matz = Object.new
def matz.speak
  "Place your burden to machine's shoulders"
end

What’s going on here is that we’re adding the speak method to matz‘s metaclass, and the matz object inherits from its metaclass and then Object. The reason this is somewhat less clear than ideal is that the metaclass is invisible in Ruby:

matz = Object.new
def matz.speak
  "Place your burden to machine's shoulders"
end
 
matz.class #=> Object

In fact, matz‘s “class” is its invisible metaclass. We can even get access to the metaclass:

metaclass = class << matz; self; end
metaclass.instance_methods.grep(/speak/) #=> ["speak"]

At this point in other articles on this topic, you’re probably struggling to keep all of the details in your head; it seems as though there are so many rules. And what’s this class << matz thing anyway?

It turns out that all of these weird rules collapse down into a single concept: control over the self in a given part of the code. Let’s go back and take a look at some of the snippets we looked at earlier:

class Person
  def name
    "Matz"
  end
 
  self.name #=> "Person"
end

Here, we are adding the name method to the Person class. Once we say class Person, the self until the end of the block is the Person class itself.

Person.class_eval do
  def name
    "Matz"
  end
 
  self.name #=> "Person"
end

Here, we’re doing exactly the same thing: adding the name method to instances of the Person class. In this case, class_eval is setting the self to Person until the end of the block. This is all perfectly straight forward when dealing with classes, but it’s equally straight forward when dealing with metaclasses:

def Person.species
  "Homo Sapien"
end
 
Person.name #=> "Person"

As in the matz example earlier, we are defining the species method on Person‘s metaclass. We have not manipulated self, but you can see using def with an object attaches the method to the object’s metaclass.

class Person
  def self.species
    "Homo Sapien"
  end
 
  self.name #=> "Person"
end

Here, we have opened the Person class, setting the self to Person for the duration of the block, as in the example above. However, we are defining a method on Person‘s metaclass here, since we’re defining the method on an object (self). Also, you can see that self.name while inside the person class is identical to Person.name while outside it.

class << Person
  def species
    "Homo Sapien"
  end
 
  self.name #=> ""
end

Ruby provides a syntax for accessing an object’s metaclass directly. By doing class << Person, we are setting self to Person‘s metaclass for the duration of the block. As a result, the species method is added to Person‘s metaclass, rather than the class itself.

class Person
  class << self
    def species
      "Homo Sapien"
    end
 
    self.name #=> ""
  end
end

Here, we combine several of the techniques. First, we open Person, making self equal to the Person class. Next, we do class << self, making self equal to Person‘s metaclass. When we then define the species method, it is defined on Person‘s metaclass.

Person.instance_eval do
  def species
    "Homo Sapien"
  end
 
  self.name #=> "Person"
end

The last case, instance_eval, actually does something interesting. It breaks apart the self into the self that is used to execute methods and the self that is used when new methods are defined. When instance_eval is used, new methods are defined on the metaclass, but the self is the object itself.

In some of these cases, the multiple ways to achieve the same thing arise naturally out of Ruby’s semantics. After this explanation, it should be clear that def Person.species, class << Person; def species, and class Person; class << self; def species aren’t three ways to achieve the same thing by design, but that they arise out of Ruby’s flexibility with regard to specifying what self is at any given point in your program.

On the other hand, class_eval is slightly different. Because it take a block, rather than act as a keyword, it captures the local variables surrounding it. This can provide powerful DSL capabilities, in addition to controlling the self used in a code block. But other than that, they are exactly identical to the other constructs used here.

Finally, instance_eval breaks apart the self into two parts, while also giving you access to local variables defined outside of it.

In the following table, defines a new scope means that code inside the block does not have access to local variables outside of the block.

mechanism method resolution method definition new scope?
class Person Person same yes
class << Person Person’s metaclass same yes
Person.class_eval Person same no
Person.instance_eval Person Person’s metaclass no

Also note that class_eval is only available to Modules (note that Class inherits from Module) and is an alias for module_eval. Additionally, instance_exec, which was added to Ruby in 1.8.7, works exactly like instance_eval, except that it also allows you to send variables into the block.

UPDATE: Thank you to Yugui of the Ruby core team for correcting the original post, which ignored the fact that self is broken into two in the case of instance_eval.

22 Responses to “Metaprogramming in Ruby: It’s All About the Self”

Very nice writeup :) A while ago I have watch (unfortunately not the live) presentation from Dave Thomas on the same subject from scotland on rails http://scotland-on-rails.s3.amazonaws.com/2A04_DaveThomas-SOR.mp4

hey yehuda, over and over again… very interesting post – love the table at the end :-) reading about ruby every free minute I totally agree with you that meta-programming is quite simple once you got the basic concept that everything is an object and is executed against a self.

On this subject, there is always _why wonderful “seeing metaclasses clearly” http://dannytatom.github.com/metaid/

Very nice explanation! I love your take on “It’s all about self.” I’ve read documentation which tries to explain the metaclass, but your code examples and explanations really make it look simple.

I’d love to hear you do a talk on “Metaprogramming in Ruby”. As much as that’s ‘been done’, I bet you could do it better.

I’ad have to say that Dave Thomas has quite a good series on this in his Metaprogramming series on PragProg: http://www.pragprog.com/screencasts/v-dtrubyom/the-ruby-object-model-and-metaprogramming

And once you realize this, you see that classes, metaclasses, modules .. etc are all unnecessary complexity. All you really need are graphs of objects sending messages to each other. See Io (http://www.iolanguage.com/) for the best example of dynamic OO at its core.

@rich Hehe… the point of my post is to show that classes and metaclasses aren’t different constructs at all!

hiya,

Just in relation to this statement “Finally, instance_eval is another way to manipulate the self of a block of code. Unlike class_eval, which sets the self to the object’s Class, instance_eval sets the self to the object’s metaclass”

It doesn’t appear to be the case in my tests:
Person = Class.new
Person.instance_eval { self } => Person

But nonetheless a ‘def’ inside an instance_eval block defines a CLASS method (not an instance method). Quite strange.

So it seems then that inside an instance_eval block the ‘def’ keyword has special behavior, it defines methods on the metaclass, even though the self is actually the class.

You can see that this is just special behavior of ‘def’ because if you use define_method instead inside a Person.instance_eval block you actually define instance methods:

Person.instance_eval { define_method(:hello) { “instance hello world” } }

versus

Person.instance_eval { def hello; “class hello world”; end }

Person.new.hello #=> “instance hello world”
Person.hello #=> “class hello world”

Yehuda, a most excellent writeup. I will be referring every new Ruby developer I mentor and meet at this article.

w00t!

~Wayne

“Also note that class_eval is only available to Modules”, keeping in mind that Class is a subclass of Module.

Yehuda, you are little wrong. instance_eval does not change self to Person’s metaclass.

I wrote an article about more detail.

http://yugui.jp/articles/846

@yugui @banisterfiend I have updated my post to reflect that instance_eval has different behavior than the other constructs, breaking up the self into “method defined on” and “methods resolved on”. Thanks for the correction!

Great article, thanks.

I initially got a handle on ruby’s metaclasses from Brian Marick’s partially completed book (chp 3):

http://www.visibleworkings.com/little-ruby/

A great resource

@wycats Right. That is why they are unnecessary and only confuse developers. I didn’t see this until I happened to discover Io. After learning Io I was using Ruby and had to (once again) consult Pickaxe so I could remember the details of Ruby’s complicated inheritance / lookup model. Staring at the page in the context of having recently learned a simpler way, it was easy to see that classes, metaclasses, modules, class variables instance variables (and even blocks) aren’t needed. All you really need are objects that inherit from each other, slots and messages.

This is a nice write up and it is consistent with Dave Thomas’ screen casts on meta-programming. The additional information on instance_eval is interesting, though can be confusing sometime.

Thanks for the knowledge and the inspiration http://jaredgrippe.com/post/246559026/javascript-like-ruby

Nice write-up, thanks, Yehuda.
I thought I understood it pretty well until I ran across the following. Why do you think the two calls to #original_some_method, one with and one without the explicit “self” receiver, behave differently?

class Someclass

def some_method=(attributes)
puts(“in the original method with #{attributes}”)
end

alias_method :original_some_method=, :some_method=

def some_method=(attributes)
puts(“in the new method with #{attributes}”)
original_some_method=(attributes) # the original_some_method isn’t called, fails silently
puts(“between calls to original_some_method”)
self.original_some_method=(attributes) # the original_some_method IS called, because of the explicit self receiver
end
end

p = Someclass.new
p.some_method=(“foo”)

# Result is:
# in the new method with foo
# between calls to original_some_method
# in the original method with foo

@les what’s happening there is that original_some_method=(attributes) is actually setting the local variable original_some_method.

Take a look at the ripper output:

> pp Ripper::SexpBuilder.new(“original_some_method=(attributes)”).parse
[:program,
[:stmts_add,
[:stmts_new],
[:assign,
[:var_field, [:@ident, "original_some_method", [1, 0]]],
[:paren,
[:stmts_add,
[:stmts_new],
[:var_ref, [:@ident, "attributes", [1, 22]]]]]]]]

You can see that there’s an “assign” to a :var_field (local variable).

> pp Ripper::SexpBuilder.new(“self.original_some_method=(attributes)”).parse
[:program,
[:stmts_add,
[:stmts_new],
[:assign,
[:field,
[:var_ref, [:@kw, "self", [1, 0]]],
:”.”,
[:@ident, "original_some_method", [1, 5]]],
[:paren,
[:stmts_add,
[:stmts_new],
[:var_ref, [:@ident, "attributes", [1, 27]]]]]]]]

Here, there’s an “assign” to self.original_some_method.

Thanks, Yehuda
OK, that explains what is happening, but why? This same behaviour is not seen if I replace the setter methods with getter methods throughout.
In the case that Ruby is simply assigning to a local variable, wouldn’t you have expected it to send the method to the (implicit) self receiver?

@les The reason is simple. The syntax ‘bareword = :foo’ is ALWAYS local assignment. It could conceivably do something fancy like lookup to see if a ‘bareword=’ method exists, but that would be potentially even more confusing given potential method_missing? implementations. As it is you just need to remember the one simple rule to use self for setters (and occasionally to avoid keyword conflicts).

s/Homo Sapien/Homo Sapiens/g; http://en.wikipedia.org/wiki/Homo_Sapiens

Really Good – you should write a book!!

Leave a Reply

Archives

Categories

Meta