Yehuda Katz is a member of the Ember.js, Ruby on Rails and jQuery Core Teams; his 9-to-5 home is at the startup he founded, Tilde Inc.. There he works on Skylight, the smart profiler for Rails, and does Ember.js consulting. He is best known for his open source work, which also includes Thor and Handlebars. He travels the world doing open source evangelism and web standards work.

Ruby is NOT a Callable Oriented Language (It’s Object Oriented)

I recently ran across a presentation entitled Python vs. Ruby: A Battle to the Death. I didn’t consider it to be a particularly fair battle, and may well reply in more detail in a later post.

However, what struck me as most worthy of explanation was the presenter’s concern about the fact that Procs are not callable via parens.

x = Proc.new { puts "HELLO" }
x() #=> undefined method `x' for #<Object:0x1001bd298><
x.call #=> "HELLO"
x[]    #=> "HELLO"

For those coming from a callable-oriented language, like Python, this seems horribly inconsistent. Why are methods called with (), while Procs are called with [].

But what’s going on here is that Ruby doesn’t have a notion of “callable”, like Python. Instead, it has a pervasive notion of Object. Here, x is an instance of the Proc class. Both call and [] are methods on the Proc class.

Designing for the Common Case: Calling Methods

Coming from a callable-oriented language, this might seem jarring. But Ruby is designed around Objects and the common cases of working with objects.

Calling methods is far more common than wanting to get an instance of Method, so Ruby optimizes the case of calling methods, with a slightly less elegant form to access an instance:

class Greeter
  def say
    puts "Hello world!"
  end
end
 
Greeter.new.say #=> "Hello world!"
 
Greeter.method(:new) #=> #<Method: Class#new>
Greeter.new.method(:say) #=> #<Method: Greeter#say>
 
# This is so that you don't have to say:
 
Greeter.new().say()

Ruby considers the common case of calling methods, and optimizes that case while still making the less common case possible.

Designing for the Common Case: How Blocks Are Really Used

One of the reasons that the Proc.new case throws off Rubyists in debates is that Rubyists literally never call Proc objects using [].

In Ruby, Procs are the object passed to methods when using block syntax. Here is how Procs are actually used:

def hello
  puts "Hello world!"
  yield
  puts "Goodbye cruel world!"
end
 
hello { puts "I am in the world!" }

When examining languages that support passing anonymous functions to functions (like JavaScript), it turns out that the vast majority of such cases involve a single anonymous function. As a result, Matz (inspired by Smalltalk) built in the idea of a block as a core construct in Ruby. In the vast majority of cases, blocks are created using lightweight syntax ({} or do/end) and called using yield.

In some cases, blocks are passed from one method to the next, before they are finally called using yield:

def step1(&block)
  puts "Step 1"
  step2(&block)
end
 
def step2
  puts "Step 2"
  yield
end
 
step1 { puts "Do the action!" } #=> "Step 1\nStep 2\nDo the action!"

As you can see, Ruby builds in the idea of calling a block into the core language. I searched through Rails (a fairly large codebase) for instances of using [] to call a Proc and while we use blocks extremely commonly, we don’t use [] to call them.

I suspect that the reason this comes up is that people who are used to having to define standalone functions, pass them around, and then call them are looking for the analogous constructs in Ruby, but are missing the different paradigm used by Ruby.

Consistent Method Execution

Fundamentally, the issue here comes down to this:

def foo
  proc {}
end
 
foo()

In Ruby, methods are invoked with our without parentheses. All methods return values, which are always Objects. All Objects have methods. So foo() is a method call that returns a Proc object. It’s extremely consistent, with very few axioms. The fact that the axioms aren’t the same as those in a callable-oriented language doesn’t make them “weird”.

38 Responses to “Ruby is NOT a Callable Oriented Language (It’s Object Oriented)”

I didn’t even know squarekets could be used to call procs.

I learned it from pythonistas a few minutes ago.

Nice article.
Indeed, the main difference lay in the way ruby coders see the implementation of Procs…
… there’s truly no need to compare Python and Ruby in that context. :-)

I think one of the things that makes Ruby what it is is the forced encapsulation of attributes. Ruby objects don’t have attributes, they only have methods that access instance variables that are hidden from the outside. This contrasts with for example JavaScript (I don’t know Python very well), where objects only have attributes, but some of those attributes (properties in JS lingo) happen to be functions, in which context they’re called methods.

I think both implementations are elegant, but we can’t have both at the same time*. So, while Ruby can’t pass functions around as easily and elegant as JavaScript, JavaScript has no built-in encapsulation and programmers have to resort to getThis and setThat methods, exposing the difference between properties and methods. And, as all good programmers know, we should encapsulate properties that are part of the object’s interface, and writing getters and setters for everything gets tiring pretty fast. Ruby’s lack of first class functions may seem a little inelegant, but all in all I think Ruby’s choice resulted in less frustration for the programmer.

* I know JavaScript now actually has both functions-as-properties and encapsulated properties through getters and setters, but this comes at the cost of added complexity and (IMO) a less elegant implementation.

Thanks for another great post. Have been learning a lot from your ruby articles.

The problem with Proc is really its name which gives the impression of much more complexity than there really is.
It’s just an object storing a block for future use.

None of the complexity which would be there in most other languages.

I’ve been waiting for a high profile Rubyist like you to comment on this. Well done!

I wasn’t shocked when Gary ‘fudged some numbers’ in the Ruby-related parts of his talk. As a Python-to-Ruby convert most of what he said didn’t sit well with me… I just don’t subscribe to the Python dogma (`import this`, etc). He’s clearly biased and only using Ruby because he has to (oh, and blocks – a missing feature in Python I don’t know you could live without).

Anyway, cheers!

By way of comparison, Scala is also fully object oriented, but the objects can be “callable”. You can treat any object as a “function” if it defines an apply method. In other words, myObj(“Hello World”) is sugar for myObj.apply(“Hello World”).

Io presents an interesting alternative. There are only Blocks (functions with assignable scope). Blocks have an activatable slot. If you set it to true, the block is activated when an object receives a message that looks up the slot with the block. If false, it returns the block.

ex.

sayHello := block(“hello” println)
sayHello //returns the block
sayHello setIsActivatable(true)
sayHello //prints “hello”
getSlot(“sayHello”) //returns the block

Actually, you can make that work, somewhat, in ruby: http://gist.github.com/310410

We may not use #[] to call procs, but using Proc#call() isn’t all that uncommon. I suspect that most Rubyists have come to the same conclusion I did WRT Proc#[], which is that it’s a confusing way to say “invoke this callable”, and semantically inconsistent with other uses of #[].

I actually like Proc#[] and use it fairly often. What I like about it is how well it duck types with a Hash or anything else that supposes normal indexing.

Same here – I use it all the time. Not as often as collect &:foo, but way more often than call. “Rubyists literally never call Proc objects using []” is literally not accurate.

Also, what you showed as the more common use case with procs, that’s not a proc, dude. That’s a block. A proc is a container object for a block, a proxy that allows you to treat code as data, even though Ruby isn’t homoiconic. Code passed to a method inside a pair of curly braces is just a plain old-fashioned block. If you called block_given? in that method, you’d get true.

To get a proc in there, you’d need:

hello lambda {puts “hello world!”}

or the obviously inferior

hello proc {puts “hello world!”}

or

hi = proc {puts “hello world!”}
hello &proc

Also note that you’re passing the value of {puts “hello world!”} to puts, where you do “puts yield”, so in fact your code doesn’t say “hello world!” at all, it says “nil.”

Heh, whoops, I mean

hello &hi

@giles a block is a Proc that goes into the “block” slot in a method:

def foo
  yield # call the Proc in the block slot
end

# call foo with proc { puts "hello" } in the block slot
foo { puts "hello" }

You could alternatively have done:

# telling ruby that the last argument should come from the block slot,
# and is not a normal parameter
def foo(&block)
  block.call # call the Proc called block
end

# call foo with proc { puts "hello" } in the block slot
foo { puts "hello" }

but these are identical. The & syntax in a method signature simply says “give a name to the Proc in the block slot”. It’s most commonly used to pass it along to another method, as I showed in my post. In that case, the & syntax in the call says “don’t pass this Proc as a parameter; instead, put it in the block slot”.

I’ve got to admit I had no idea that procs can be passed like that:

def step1(&block)
puts “Step 1″
step2(&block)
end

and I find the ampersand very confusing in the step2 method invocation

My first thought was to use this instead:

def step1(&block)
puts “Step 1″
step2 {block.call}
end

Sorry, cant edit previous post:

and in a way this approach, ie.

def step1(&block)
puts “Step 1″
step2 {block}
end

Allows for some flexibility, since we can in step2:

def step2
puts “Step 2″
yield.call #directly call the proc
x = yield
step_other {x} # or assign the proc to a variable and move it along again
end

@wycats no @giles is right.

Procs aren’t instantiated when a block is passed to a method which yields to it.

From Flanagan and Matsumoto “The Ruby Programming Language” p176

“Blocks, like methods, are not objects that Ruby can manipulate. But it’s possible to create an object that represents a block, and this is actually done with some frequency in Ruby programs. A Proc object represents a block. Like a Method object, we can execute the code of a block through the Proc which represents it.”

In order to create a Proc object you need to either pass a block to a method which declares a special argument with an & preceding the name in the method definition, which has the effect of creating the proc object and assigning it to that argument, or pass a block to Proc.new, or to Kernel#lambda, or to Kernel#proc, or in 1.9 by using the new -> lambda literal syntax.

Kernel#lambda and the lambda literal both produce instances of Proc with some subtle differences in how they handle control flow statements like return and break, and the semantics of how arguments are bound when the proc/lambda is called.

One major difference between blocks and procs is that procs are closures, they retain the bindings of any variables visible to them, even after the method which created them returns, and I believe that this is why procs are NOT created in a case like

def foo
   yield
end

foo {puts "Hi"}

Because there’s no need to capture the bindings, the yield MUST happen by definition before the foo method returns, so there’s no need for the overhead of creating a closure.

Also, I think the connection between Smalltalk blocks and Ruby blocks/procs/lambdas is a bit more tenuous than you suggest.

Smalltalk has (or at least had at the time we developed the X3J20 standard) no yield equivalent. There is no hidden special/environmental block parameter to a Smalltalk method. Blocks were just normal method arguments. This also meant that a Smalltalk method can have zero, one, or many block arguments.

Blocks were invoked by sending them a message like value (for a block with no arguments), value: (for a block with one argument), value:value: (two arguments) etc.

The Ruby yield IS very useful in Ruby’s implementation of enumeration methods, most of which do appear to be modeled after methods in the Smalltalk collection classes. For example, in Smalltalk you might write

   aCollection inject: 0 into: [:sum :element | sum + element]

Ruby yield effectively optimizes this, by not creating a closure.

But another major use of blocks in Smalltalk is to implement control flow, the Ruby fragment:

   if expression
       "True, too true!"
   else
      "Liar, liar, pants on fire"
   end

would be written in Smalltalk as:

   expression ifTrue: ['True, too true!'] ifFalse: ['Liar, liar, pants on fire']

The expression would normally evaluate to a boolean (i.e. a singleton instance of either TrueClass, or FalseClass) and the implementations of ifTrue:ifFalse: would look like this

"In class TrueClass"
ifTrue: trueBlock ifFalse: falseBlock
    ^trueBlock value

"In class FalseClass"
ifTrue: trueBlock ifFalse: falseBlock
    ^falseBlock value

Doing this in Ruby is possible to a degree but requires explicitly creating procs/lambdas for the additional block arguments.

On the other hand, although Smalltalk works this way conceptually, most implementations cheated and compiled methods like ifTrue:ifFalse: to control flow bytecodes in the methods in which the ‘invocations’ appeared, and either required the receiver of these methods to actually be a boolean, or treated the case where they weren’t as an exceptional condition handled by the VM.

But getting back to the original pythonista argument. The gripe that you can’t call a block/proc with parens is just quibbling about the flavor of the syntactic sugar, and there’s more than enough similar things in Python which might be tasty to a Python fan don’t fit well with me, like the need to explicitly declare a self parameter in methods.

Oops, I was afraid that I’d lose control over the indentation in that last comment, and I did, hopefully it still makes some sense.

I don’t see anyone mentioning that the method analogue to the original example is

def five; 5 end
x = method(:five)

in which case calling it is exactly the same as a Proc

x.call
=> 5
x[]
=> 5

It looks like the presenter didn’t have a clear idea of what the code was doing and is just criticizing Ruby for not passing and calling methods and procs identically to how Python treats functions.

@dorian absolutely. Ruby is *optimized* for the invoking case, providing a less elegant syntax for getting a callable. Python is *optimized* for the “passing a callable” case, providing a less elegant syntax for invoking a method. Considering the relative frequency of each, it’s clear (at least to me) that a more elegant syntax for invoking methods wins out over a more elegant syntax for retrieving an object representing the method that can be passed around.

@rick thanks for that long response. I personally consider the difference between blocks and procs to be equivalent to the fact that Objects don’t get metaclasses until you ask for them. In other words, while it may be a true optimization that (one or more) implementations of Ruby perform, it has no effect on the way a Ruby programmer should think about problems.

In this case, nothing is *lost* by the Ruby programmer considering all blocks to be instances of the Proc class, and the instant he does anything to examine the situation, the illusion is preserved. In other words, I think Flanagan and Matz, in this case, are exposing internals of Ruby that are completely irrelevant to the Ruby language.

Here’s a mind game: could you imagine a Ruby implementation that created a Proc for every block. Would the difference between this implementation and CRuby be detectable in any way?

I think there is also a valid reason for this which relates to syntax itself: procs with no arguments will cause an ambiguity when being called. Here is an example:

p = lambda{ ‘hello’}

if Ruby to allow direct calls on blocks, then how should it respond to this?

puts p

is that p.to_s or p() ?

So the problem is that procs with no arguments will confuse the interpreter, and that’s why the above call will default to p.to_s

=>#

If you want to call it so: either .call or [] or even .() in ruby 1.9
puts p.() #=> hello

Why does everyone insist that Object Oriented Nature of Ruby is the reason of all its characteristics?
There is no such thing (that i know) as a “callable oriented language”, and if it existed anyway, it didn’t have to go against Object Orientation.

Maybe a unification of Procs, Blocks and Lambdas could lead to less confusion from people coming from other languages. at the end it seems that the only difference is how to call this “executable” constructs on the language.

History has proven that simplification leads to more robust solutions.

I watched the same presentation as a pythonista and I thought it actually expressed a pythonista’s respect and yearning for code blocks quite well (http://twitter.com/dnene/status/9441446788). I am not sure what you considered unfair in the talk so will look forward to your thoughts on the same. But I guess what it boils down to is in python functions and function references are first class citizens. Isn’t that an easier way to express rather than coining a new term “Callable Oriented” ? Or did I not understand the distinction well enough.

here’s a link to the actual video of the ‘Python vs. Ruby: A Battle to the Death’ presentation:

http://vimeo.com/9471538

“with our without parentheses” should be “with or without parentheses”

http://www.emendapp.com/sites/yehudakatz.com/edits/1

nothing is *lost* by the Ruby programmer considering all blocks to be instances of the Proc class

except accurate semantics, which seem to be the whole goal of your post. aren’t you trying to correct a Pythonista’s inaccurate description of Ruby semantics? wouldn’t accuracy be useful in that context?

Here are some other “accurate” things:

When you use =~, there is no actual MatchData object, $1 just behaves like it was indexing into an implicit MatchData

Objects do not have metaclasses until you ask for one. A bare object is hooked directly up to it’s class; when you modify its metaclass (like class << or extend) the class pointer is repointed at a metaclass that is only now created.

What all three of these cases have in common is that the details have nothing whatsoever to do with the semantics of Ruby and are purely internal Ruby optimizations (which may or may not exist on other implementations). In cases where a construct is not a Ruby object in MRI’s internals, but becomes one on demand, the construct is semantically equivalent to the object, and we are polluting Ruby’s semantics by exposing internal implementation details as semantically meaningful.

@wycats I was also unaware of the Proc#[] syntax, thanks for the info.

It certainly makes some sense to optimize the syntax for calling methods (without parens). It is an interesting choice, to optimize for the common case rather than providing lower-level constructs that can be used in many ways.

In an unrelated example, I was translating a small Clojure snippet to Ruby and Python. In Ruby I found the standard libraries provide the exact solution to the problem, but building up lower-level constructs towards the solution was awkward.
http://gist.github.com/190489
(if there is a better way, please do fork my gist)

Python’s adoption of Haskell-style comprehensions proved far more powerful.

Ruby is a nice language, but what I am gathering is that Ruby unsurprising fits with a more OOP mindset than a mindset of functional-style composition.

Hey Nathan,

I’ve forked your gist at http://gist.github.com/313605 and improved the code, leveraging Enumerators, which were added to 1.8.7+ specifically to solve this sort of problem. Also, for performance (which doesn’t seem to matter much in this case), you might want to convert the predicate Array into a Set first, which will make the subsequent #include? lookups simple Hash lookups (reducing O(m*n) to O(n)).

In cases where a construct is not a Ruby object in MRI’s internals, but becomes one on demand, the construct is semantically equivalent to the object, and we are polluting Ruby’s semantics by exposing internal implementation details as semantically meaningful.

That’s great but you didn’t answer my questions. If you’re criticizing a Pythonista for failing to understand Ruby, after that Pythonista made an over-generalized assumption about how the language worked based on how the language appeared to work, I don’t see how you can then argue in favor of generalizing away how the language works and saying it’s more important to pay attention to how the language appears to work. How do you justify this? Don’t you understand that you appear to be taking both sides of the issue simultaneously?

Thanks Yehuda, your version is much cleaner. Obviously I still have a fair bit to learn about Ruby.

Thought I’d weigh in on the difference between blocks and Procs, having implemented them a few times. There is no semantic concept of a block, there is only the syntax. All semantics related to blocks are within the Proc object.

Rick mentions something about them being different because yield can not capture locals:

One major difference between blocks and procs is that procs are closures, they retain the bindings of any variables visible to them, even after the method which created them returns…

This is just wrong. The lexical closure in both cases is exactly the same. The only difference is that the closure would have a shorter GC lifetime since there is only one reference to the closure. But thats entirely GC related, nothing to do with the closure itself.

Whether or not there is any internal optimization/laziness related to allocation of Proc objects is beside the point, because they are exposed consistently. This policy is the same as what occurs with meta classes.

For those looking to say that there is a difference between a block and a Proc, I’d ask you to demonstrate it. I’m happy to discuss any differences you put forth.

Yehuda… Would you agree that:
foo = Proc.new { ‘foo’ }
foo()

is prettier than:
foo.call

or

foo.()
?

In that regard, Python clearly wins. It just does. But, so what?

@Mario if you actually read Yehuda’s post you would know the argument was not about which was ‘prettier’ it was about ‘consistency’! :)

@banisterfiend, I am not sure I agree about the consistency argument, though perhaps I just need to think about it more. A proc is not exactly an unattached method and a lambda is closer to being a method, but still isn’t. Yet I would like to be able to call them like methods (foo()). I always assumed that we couldn’t because of a parser limitation (or something like that). I would find it more consistent if procs and lambdas were consistent with methods (albeit unattached ones) in how they can be called, though perhaps because they are unattached they should not “be” like methods because they really aren’t methods. That said, my preference is for one syntax over another and it’s just a preference. I probably shouldn’t have said one wins over another because now that I read it, it comes across as a flame, which I didn’t mean.

@Mario, procs and lambdas are *objects*. methods are not objects (do not confuse methods with instances of Method). It therefore makes sense that you can only invoke a proc/lambda by sending it a message (#call or #.()).

Nonetheless in some languages (i.e Scala) you can invoke method objects by using (), but in Ruby this operator can not be overloaded, probably because parentheses are optional in Ruby.

Thanks for the clarification, a fair article. Also a nice point about distiction among OO languages, they are not all the same, there are different principles.

I’ve also written a few words with code examples about ruby blocks and closures: http://blog.mostof.it/why-ruby-part-two-blocks-and-closures/

Leave a Reply

Archives

Categories

Meta