Announcing Handlebars.js

For a number of years, I've been about as active in client-side development (through jQuery) as I have been active in server-side development (through Merb and then Rails). Recently, I've released a couple of libraries (jQuery Offline and Rack::Offline), designed to make it easier to build applications that can withstand both flaky connections and a complete lack of connectivity. Of course, those libraries leveraged features in modern browsers, and have gotten a bunch of use.

For jQuery Offline, the basic strategy is that instead of requesting a new JSON object before you can show anything on the page, you first try to retrieve that data from cache, show some stale content. In parallel, you'd download the new content (if possible), and update the display with the content in question. You've probably seen this pattern at work in applications like the official Twitter application for the iPhone, which shows stale content when you first boot the app, and downloads new content if you have Internet connectivity in the background.

To take the JSON and convert it into some HTML for the page, I recommended jquery-tmpl or mustache.js. Both of these libraries allow you to take a JSON object and convert it on the fly into HTML, which you can insert into the page. Both libraries are still very good, and I could easily imagine problem-free uses for both of them.

Of the two libraries, I really liked Mustache's approach. Mustache envisions a template as something that is completely code-free, and populated via a JavaScript Object. This Object can contain normal values, or functions which are used as helpers.

<script type="text/x-mustache-template" name="posts">  
{{#posts}}
<h1><a href="{{url}}">{{title}}</a></h1>

{{#div}}
{{body}}
{{/div}}
{{/post}}
</script>  

You'd then populate this template with a JavaScript object:

var view = {posts: [  
  { id: 1,
    url: "/posts/1",
    title: "Hello world!",
    div: function(text, render) { return "<div class='post entry'>" + render(text) + "</div>" },
    post: "Hello world. This is my first post"
  },
  { id: 2,
    url: "/posts/2",
    title: "Goodbye world!",
    div: function(text, render) { return "<div class='post entry'>" + render(text) + "</div>" },
    post: "Goodbye world. Sadly, this is going to be my last post"
  }
]};

var template = $("script[name=posts]").html();  
var html     = Mustache.to_html(template, view)  

The basic idea here is that the template itself is completely free of code, and just uses keys from the JavaScript Object to convert it into HTML. The JavaScript Object can have some functionality, and even code, to help the template along. I really like the idea of limiting the template to referencing an element of the JavaScript object, and completely firewalling any kind of code to the view object. Among other things, this allows a single template to run in both Ruby on the server-side and JavaScript on the client side.

When using Mustache, I ran into a few issues. One minor issue was that the base implementation of mustache.js was essentially interpreted, rather than compiled. This means that the implementation is slower than you might expect of a template language, especially one as limited as Mustache.

More concerningly, the base Mustache syntax was too limited for a lot of things I wanted to use it for. I wasn't immediately sure if it was just me, so I asked a bunch of people about their opinions of Mustache. One of the people who replied was Alan Johnson, who had been using Mustache a lot and volunteered his thoughts. Without much prompting, Alan's concerns about the limitations of Mustache (in both our cases, born from real attempts to use it) closely mirrored mine. My thinking validated for the moment, I got to work trying to extend Mustache to get around these limitations.

Before I continue, here are some of the more important problems I had:

  • Function values (like div above) needed to be located in the same place in the hierarchy as object being used as a context. This made providing a set of global helpers (like a framework would want to do) quite difficult.
  • Even if the syntax was modified to make function helpers global, the functions would not have access to the current context, so they would be of limited value (imagine trying to change the url key above into a helper)
  • I commonly wanted access to a key in a parent scope when working in a child scope. For instance, if a post had many comments, I would sometimes want access to the post's title (or other details) while rendering the child

There were other small irritations, but these were definitely the major ones. A lot of the people I spoke to solved these problems by rewriting the JavaScript Object before passing it into the Mustache system. Some of my initial thinking, in fact, revolved around making that sort of rewriting easier. In the end, though, I wanted to extend the Mustache syntax with a few new features that would make it behave more like what I expected.

From the beginning, I wanted my extensions to be a strict superset of the official Mustache manual, which is pretty clear in some areas and somewhat more vague in other areas. I really like the Mustache syntax and philosophy, and wanted to make sure whatever I came up with would not diverge philosophically from what I enjoyed about Mustache itself.

Compiled, Not Interpreted

The first step was to make the Mustache syntax compiled, not interpreted. There is another project (called Mu) which is a compiled variant of Mustache, but it was explicitly built for Node.js, and is built around server-side JavaScript, rather than client-side JavaScript. I also wanted to build it with some of the extensions I wanted to add in mind from the beginning.

The parser/compiler is a lot like the ERB compiler, and has a lot of what you'd expect from a parser/compiler (getChar, peek, addText, addExpression, etc.). I designed it from the beginning to be extended, taking some of the architecture from Erubis, which separates out how different kinds of tags are added to the source it is building up. We'll probably have to make a few tweaks before extensions can be easily dropped in, but I already have a couple in mind that will help flesh out the API.

Extended Paths

The first extension to Mustache is "extended paths". In Mustache, paths are always relative to the current context (which can change with iteration), and are always a single key. For instance, the url path from the above example references the url key in the current post that Mustache is iterating over.

Extended paths allow you to go deeper into the Object (byline/name) or up the hierarchy (../title). This doesn't diverge from the core Mustache philosophy of avoiding code in templates, but it does allow templates to more flexibly reference other parts of the Object, apart from the context they are in the process of rendering. A lot of people implement this feature by rewriting the Object before passing it into the template--Handlebars allows that rewriting to happen on the fly.

Global Lambdas with Access to the Current Context

Mustache allows the Object to contain lambdas, but they must appear in the same context as the rest of the keys being rendered. If iterating over a list of projects, for instance, each project must have a key referencing the lambda, even if it's always the same. Handlebars allows you to provide a global Object of lambdas, which it will look in when it does not find a key in the current context. This allows you to create helpers to use throughout your Template. Again, people implemented this feature by rewriting the Object, and Handlebars is just making it possible to achieve the same goal without that rewriting.

After adding global lambdas, those helpers might want to do different things based on the context. Handlebars invokes its helper functions with a this that reflects the current context. We can rewrite the above example as:

var view = {posts: [  
  { id: 1,
    title: "Hello world!",
    div: function(text, render) { return "<div class='post entry'>" + render(text) + "</div>" },
    post: "Hello world. This is my first post"
  },
  { id: 2,
    title: "Goodbye world!",
    div: function(text, render) { return "<div class='post entry'>" + render(text) + "</div>" },
    post: "Goodbye world. Sadly, this is going to be my last post"
  }
]};

var helpers = {  
  url: function() {
    return "/posts/" + this.id;
  }
}

Of course, this example is pretty simple, and you could have just done {{title}}, but this illustrates that helpers can use the current context to control the output.

Helpers Take a Context

Helpers could be made even more generic if they could take a context (referenced as a path) as an argument. As I've shown above, helpers are always executed with their current rendering context as this, but it could be convenient to pass a different context, referenced by its path, to the function.

{{#projects}}
  <p>Back: {{url ..}}</p>

  <div>
    {{body}}
  </div>
{{/project}}

In this example, we want each project to include a link back to the parent project. By passing in a context (in this case, the object referenced by the extended path ..) we can make our helpers even more generic.

var helpers = {  
  url: function(context) {
    return "/" + context.type + "/" + context.id;
  }
}

Block Helpers

The Mustache manual is a bit light on how lambda functions should work, referencing a render method that it doesn't pass into the function. I wanted block helpers to take a function that was essentially identical to the compiled function for the entire template, and would be populated in the same way a regular template gets populated. A good example use for this type of helper is a custom list.

<script type="text/x-handlebars-template" name="items">  
{{#list items}}
<h1>{{title}}</h1>  
<div>{{body}}</div>  
{{/list}}
</script>  
var view = { items:  
  [
    { title: "Hello world",         body: "This is my first post" },
    { title: "Goodbye cruel world", body: "This will be my last post ;(" }
  ]
}

var helpers = {  
  list: function(context, fn) {
    var buffer = "<ul class='list'>\n";
    for(var i=0, j=context.length; i < j; i++) {
      buffer.push( "<li>" + fn(context[i]) + "</li>\n" )
    }
    buffer.push("</ul>");
  }
}

var source   = $("script[name=items]").html();  
var template = Handlebars.compile(source);

template(view, helpers)  

This will output:

<ul class='list'>  
<li><h1>Hello world</h1>  
<div>This is my first post</div></li>  
<li><h1>Goodbye world</h1>  
<div>This will be my last post ;(</div></li>  
</ul>  

As you can see, block helpers can now invoke the sub-template many times with different contexts. And block helpers, like regular helpers, can receive a context, expressed by an extended path, as a parameter.

Iteration and Booleans Are Just Block Helpers

One thing that bugged me was that iteration and booleans were special-cased in the normal Mustache specification. I didn't want to change the behavior of iteration or booleans, but I did want to make them behave the same as regular block helpers. By making block helpers more powerful, I was able to achieve this with one small additional change.

In particular, rather than hardcode the behavior of iteration and booleans, I created a new feature allowing a particular function to be used when a block helper could not be found. This way, every use of {{#foo}} was a block helper, but iteration was just invoking the special helperMissing block helper. In both this case and the case of conditionals, the missing helper was treated as a context to be looked up as a path.

The helperMissing function itself is quite simple:

Handlebars.helperMissing = function(object, fn) {  
  var ret = "";

  if(object === true) {
    return fn(this);
  } else if(object === false) {
    return "";
  } else if(toString.call(object) === "[object Array]") {
    for(var i=0, j=object.length; i < j; i++) {
      ret = ret + fn(object[i]);
    }
    return ret;
  } else {
    return fn(object);
  }
};

First, if the object referenced is true, we invoke the template block, using the current context. If the object referenced is false, we return an empty String.

If the object referenced is an Array, we iterate over the elements of the Array, invoking the template block once in the context of each element of the Array. This looks a lot like the items block helper we created above.

Finally, if the object referenced is anything else, we invoke the template block in the context of that object. Because we made block helpers more powerful, it is easy to implement the hardcoded boolean, iteration and context-switching behavior of the Mustache spec in terms of the block helpers. Note that if you wanted to change this behavior, you would just need to override Handlebars.helperMissing with a new function of your own design.

Better Inverted Sections

Finally, Mustache allows the specification of inverted sections, which behave like the reverse of a block. For instance, where {{#boolean}} will run the block if boolean is true (or a non-falsy value), and skip it if boolean is false, {{^boolean}} will run the block if boolean is false, and skip it if boolean is true (or non-falsy value).

A very common usage of inverted sections is to behave like if/else blocks:

{{#project}}
<h1>{{name}}</h1>  
<div>{{body}}</div>  
{{/project}}
{{^project}}
<h1>No projects</h1>  
{{/project}}

Because of how common this is, Handlebars has a shortened syntax for this case:

{{#project}}
<h1>{{name}}</h1>  
<div>{{body}}</div>  
{{^}}
<h1>No Projects</h1>  
{{/project}}

Additionally, you can specify the inverse form of a custom block helper by assigning a not property on the helper function.

{{#list projects}}
<h1>{{name}}</h1>  
<div>{{body}}</div>  
{{^}}
<h1>No Projects</h1>  
{{/project}}
var helpers = {  
  list: function(items, fn) {
    if(!items.length) return "";

    var buffer = "<ul>";
    for(var i=0, j=items.length; i < j; i++) {
      buffer += fn(items[i]);
    }
    buffer += "</ul>"
    return buffer;
  }
}

helpers.list.not = function(items, fn) {  
  if(!items.length) { return fn(); }
}

Wrapping Up

Long story short: the goal of handlebars.js is to create a syntax-compatible superset of Mustache that solves a lot of problems I've had when working with Mustache (especially as part of a larger library).

I want to personally thank Alan Johnson for contacting me early and fleshing out my initial prototype into a workable, releasable library. His experience with Mustache, as well as his willingness to spend time on the nitty gritty details really made the difference. Thanks!

As always, please submit bug reports to the GitHub Issues tracker associated with the project. I really love the new GitHub pull request system, so please submit patches using it.