Yehuda Katz is a member of the Ember.js, Ruby on Rails and jQuery Core Teams; his 9-to-5 home is at the startup he founded, Tilde Inc.. There he works on Skylight, the smart profiler for Rails, and does Ember.js consulting. He is best known for his open source work, which also includes Thor and Handlebars. He travels the world doing open source evangelism and web standards work.

Archive for the ‘JavaScript’ Category

A Proposal for ES.next Proposals

Over the past few years, I have occasionally expressed frustration (in public and private) about the process for approving new features to the next edition of ECMAScript. In short, the process is extremely academic in nature, and is peppered with inside baseball terms that make it nearly impossible for lay developers to provide feedback about proposed new features. In general, this frustration was met with a general assumption that the current process works the way it does for good reason, and that academic descriptions of the new features was the correct (and only) way to properly discuss them.

I have nothing against new features being described in the language of implementors, but I would like to propose some additions to the current process that would make it significantly easier for language consumers (especially framework and library implementors) to provide timely feedback about the proposal in the hope of making an impact before it’s too late.

I would like proposals for new features to have the following elements, in addition to whatever elements are already normally included (such as BNF for any new syntax).

What Problem Are We Solving?

At a high level, what can language users do now that they could not do before. In some cases, proposals may provide simpler or more convenient ways to achieve already-possible goals. These kinds of proposals are often just as important. For example, there is a current proposal to provide a new syntax for class creation. In this case, the new class syntax significantly improves the experience of building a common JavaScript construct.

Provide Non-Trivial Example Code Showing the Problem

If the proposal is solving a problem that exists in the wild, it should be possible to identify non-trivial examples of the problem rearing its head. At the very least, the process of identifying or synthesizing these examples will help language users understand what real-world problems the proposal is attempting to solve. At best, finding real-world examples will help refine the proposal at an early stage.

Show How the Example Code is Improved With the Proposal

After identifying or synthesizing example code to illustrate the problem, show how the problem would be improved if the proposal was accepted. In some cases, the problem is as simple as “needing a large library to perform this operation” and the solution is “building common functionality into the language”. In an example from a related field, the DOM library, the problem addressed by querySelectorAll was “many incompatible implementations of a CSS3 selector library”. The solution was to build the functionality into the browser.

In this case, a mistake in the querySelectorAll specification, which was resolved by the addition of queryScopedSelectorAll in the Selectors API Level 2 could have been addressed ahead of time by evaluating real-life code using selector engine libraries. Of course, the DOM library is not the same as the language specification, so the example is merely an analogy.

What are Alternate ES3 (or ES5) Solutions to the Same Problem?

Simply enumerate the ways that existing JavaScript developers have attempted to resolve the problem without language support. In small language changes, this overlaps considerably with the previous questions. By having proposal authors enumerate existing solutions to the problem, it will be easier for language users to identify the scope of the solution.

This will allow language users to provide feedback about how the solution on offer stacks up compared to existing pure-JS solutions.

Are there any restrictions that do not exist in original pure-JS solutions?

Are there any restrictions in the proposal that limit its utility as a solution to the problem in question, especially if those restrictions do not apply to solutions currently used by language users.

If the Proposal is For New Syntax, What if Anything Does it Desugar To?

Also, if the proposal desugars, why choose this particular desugaring as opposed to alternatives?

If New Syntax, Can it Be Compiled to ES3 (or ES5)?

If the proposal can desugar to an older version of the specification, can a source-to-source translator be written? If so, is there a reference implementation of a source-to-source translator written in that version?

By writing such a source-to-source translator, existing language users can experiment with the new syntax easily in a browser environment without requiring a separate compilation pass. This also allows users to build an in-browser translation UI (similar to try CoffeeScript), which can improve general understanding of the new syntax and produce important feedback.

To be more specific, what I would like to see here is a general-purpose source-to-source translation engine written in ES3 with a mechanism for plugging in translation passes for specific features. If new features come with translation passes, it would be trivial for language users to try out new features incrementally in production applications (with a nice development-time workflow). This would provide usability feedback at an early enough stage for it to be useful.

If a New Standard Library, Can it Be Polyfilled to ES3 (or ES5)?

If the proposal is for a new library whose syntax is valid in an earlier version of the specification, can it be implemented in terms of the existing primitives available in that version. If necessary, primitives not defined by the language, but provided historically by web browsers, can be used instead. The goal is to provide shims for older browsers so that a much broader group of people can experiment with the APIs and provide feedback.

In general, new libraries that parse in older versions of the specifications should also come with a well-defined mechanism to detect whether the feature is present, to make it easy for library and framework vendors, as well as the general public, to opt their users into the new features where appropriate.

Even if a fully backwards-compatible shim cannot be provided, it is still useful to provide a partial implementation together with a feature detection mechanism. At the very least, error-checking shims can be useful, so language users can easily understand the interface to the proposed library.

Does the Proposal Change any Existing Semantics?

In some cases, the proposal unavoidably changes existing semantics. For example, ES5 changed the semantics of an indirect call to eval (a call to an alias to eval, such as x = eval; x('some code') to use the global environment for the evaluated code. In ES3, indirect calls to eval behaved the same as direct calls to eval.

These cases are rare, and in most cases, require an explicit opt-in (such as the directive "use strict";).

When such changes are made, especially when they do not require an opt-in, they should be explicitly called out in the proposal to gather feedback about their likely impact on existing code. Even when they require an opt-in, information about the frequency of their use could be useful to assess the difficulty of opting in.

Since these opt-ins often enable new features as well as changing existing semantics, understanding the impact of the opt-in on existing code would help language users assess their overall utility and timeframe for adoption. This information could also help drive these decisions.

Understanding “Prototypes” in JavaScript

For the purposes of this post, I will be talking about JavaScript objects using syntax defined in ECMAScript 5.1. The basic semantics existed in Edition 3, but they were not well exposed.

A Whole New Object

In JavaScript, objects are pairs of keys and values (in Ruby, this structure is called a Hash; in Python, it’s called a dictionary). For example, if I wanted to describe my name, I could have an object with two keys: `firstName` would point to “Yehuda” and `lastName` would point to “Katz”. Keys in a JavaScript object are Strings.

To create the simplest new object in JavaScript, you can use Object.create:

var person = Object.create(null); // this creates an empty objects

Why didn’t we just use var person = {};? Stick with me! To look up a value in the object by key, use bracket notation. If there is no value for the key in question, JavaScript will return `undefined`.

person['name'] // undefined

If the String is a valid identifier[1], you can use the dot form:

person.name // undefined

[1] in general, an identifier starts with a unicode letter, $, _, followed by any of the starting characters or numbers. A valid identifier must also not be a reserved word. There are other allowed characters, such as unicode combining marks, unicode connecting punctuation, and unicode escape sequences. Check out the spec for the full details

Adding values

So now you have an empty object. Not that useful, eh? Before we can add some properties, we need to understand what a property (what the spec calls a “named data property”) looks like in JavaScript.

Obviously, a property has a name and a value. In addition, a property can be enumerable, configurable and writable. If a value is enumerable, it will show up when enumerating over an object using a for(prop in obj) loop. If a property is writable, you can replace it. If a property is configurable, you can delete it or change its other attributes.

In general, when we create a new property, we will want it to be enumerable, configurable, and writable. In fact, prior to ECMAScript 5, that was the only kind of property a user could create directly.

We can add a property to an object using Object.defineProperty. Let’s add a first name and last name to our empty object:

var person = Object.create(null);
Object.defineProperty(person, 'firstName', {
  value: "Yehuda",
  writable: true,
  enumerable: true,
  configurable: true
});
 
Object.defineProperty(person, 'lastName', {
  value: "Katz",
  writable: true,
  enumerable: true,
  configurable: true
});

Obviously, this is extremely verbose. We can make it a bit less verbose by eliminating the common defaults:

var config = {
  writable: true,
  enumerable: true,
  configurable: true
};
 
var defineProperty = function(obj, name, value) {
  config.value = value;
  Object.defineProperty(obj, name, config);
}
 
var person = Object.create(null);
defineProperty(person, 'firstName', "Yehuda");
defineProperty(person, 'lastName',   "Katz");

Still, this is pretty ugly to create a simple property list. Before we can get to a prettier solution, we will need to add another weapon to our JavaScript object arsenal.

Prototypes

So far, we’ve talked about objects as simple pairs of keys and values. In fact, JavaScript objects also have one additional attribute: a pointer to another object. We call this pointer the object’s prototype. If you try to look up a key on an object and it is not found, JavaScript will look for it in the prototype. It will follow the “prototype chain” until it sees a null value. In that case, it returns undefined.

You’ll recall that we created a new object by invoking Object.create(null). The parameter tells JavaScript what it should set as the Object’s prototype. You can look up an object’s prototype by using Object.getPrototypeOf:

var man = Object.create(null);
defineProperty(man, 'sex', "male");
 
var yehuda = Object.create(man);
defineProperty(yehuda, 'firstName', "Yehuda");
defineProperty(yehuda, 'lastName', "Katz");
 
yehuda.sex       // "male"
yehuda.firstName // "Yehuda"
yehuda.lastName  // "Katz"
 
Object.getPrototypeOf(yehuda) // returns the man object

We can also add functions that we share across many objects this way:

var person = Object.create(null);
defineProperty(person, 'fullName', function() {
  return this.firstName + ' ' + this.lastName;
});
 
// this time, let's make man's prototype person, so all
// men share the fullName function
var man = Object.create(person);
defineProperty(man, 'sex', "male");
 
var yehuda = Object.create(man);
defineProperty(yehuda, 'firstName', "Yehuda");
defineProperty(yehuda, 'lastName', "Katz");
 
yehuda.sex        // "male"
yehuda.fullName() // "Yehuda Katz"

Setting Properties

Since creating a new writable, configurable, enumerable property is pretty common, JavaScript makes it easy to do so using assignment syntax. Let’s update the previous example using assignment instead of defineProperty:

var person = Object.create(null);
 
// instead of using defineProperty and specifying writable,
// configurable, and enumerable, we can just assign the
// value directly and JavaScript will take care of the rest
person['fullName'] = function() {
  return this.firstName + ' ' + this.lastName;
};
 
// this time, let's make man's prototype person, so all
// men share the fullName function
var man = Object.create(person);
man['sex'] = "male";
 
var yehuda = Object.create(man);
yehuda['firstName'] = "Yehuda";
yehuda['lastName'] = "Katz";
 
yehuda.sex        // "male"
yehuda.fullName() // "Yehuda Katz"

Just like when looking up properties, if the property you are defining is an identifier, you can use dot syntax instead of bracket syntax. For instance, you could say man.sex = "male" in the example above.

Object Literals

Still, having to set a number of properties every time can get annoying. JavaScript provides a literal syntax for creating an object and assigning properties to it at one time.

var person = { firstName: "Paul", lastName: "Irish" }

This syntax is approximately sugar for:

var person = Object.create(Object.prototype);
person.firstName = "Paul";
person.lastName  = "Irish";

The most important thing about the expanded form is that object literals always set the newly created object’s prototype to an object located at Object.prototype. Internally, the object literal looks like this:

prototype-chain.png

The default Object.prototype dictionary comes with a number of the methods we have come to expect objects to contain, and through the magic of the prototype chain, all new objects created as object literal will contain these properties. Of course, objects can happily override them by defining the properties directly. Most commonly, developers will override the toString method:

var alex = { firstName: "Alex", lastName: "Russell" };
 
alex.toString() // "[object Object]"
 
var brendan = {
  firstName: "Brendan",
  lastName: "Eich",
  toString: function() { return "Brendan Eich"; }
};
 
brendan.toString() // "Brendan Eich"

This is especially useful because a number of internal coercion operations use a supplied toString method.

Unfortunately, this literal syntax only works if we are willing to make the new object’s prototype Object.prototype. This eliminates the benefits we saw earlier of sharing properties using the prototype. In many cases, the convenience of the simple object literal outweighs this loss. In other cases, you will want a simple way to create a new object with a particular prototype. I’ll explain it right afterward:

var fromPrototype = function(prototype, object) {
  var newObject = Object.create(prototype);
 
  for (var prop in object) {
    if (object.hasOwnProperty(prop)) {
      newObject[prop] = object[prop];      
    }
  }
 
  return newObject;
};
 
var person = {
  toString: function() {
    return this.firstName + ' ' + this.lastName;
  }
};
 
var man = fromPrototype(person, {
  sex: "male"
});
 
var jeremy = fromPrototype(man, {
  firstName: "Jeremy",
  lastName:  "Ashkenas"
});
 
jeremy.sex        // "male"
jeremy.toString() // "Jeremy Ashkenas"

Let’s deconstruct the fromPrototype method. The goal of this method is to create a new object with a set of properties, but with a particular prototype. First, we will use Object.create() to create a new empty object, and assign the prototype we specify. Next, we will enumerate all of the properties in the object that we supplied, and copy them over to the new object.

Remember that when you create an object literal, like the ones we are passing in to fromPrototype, it will always have Object.prototype as its prototype. By default, the properties that JavaScript includes on Object.prototype are not enumerable, so we don’t have to worry about valueOf showing up in our loop. However, since Object.prototype is an object like any other object, anyone can define a new property on it, which may (and probably would) be marked enumerable.

As a result, while we are looping through the properties on the object we passed in, we want to restrict our copying to properties that were defined on the object itself, and not found on the prototype. JavaScript includes a method called hasOwnProperty on Object.prototype to check whether a property was defined on the object itself. Since object literals will always have Object.prototype as their prototype, you can use it in this manner.

The object we created in the example above looks like this:

prototype-chain-2.png

Native Object Orientation

At this point, it should be obvious that prototypes can be used to inherit functionality, much like traditional object oriented languages. To facilitate using it in this manner, JavaScript provides a new operator.

In order to facilitate object oriented programming, JavaScript allows you to use a Function object as a combination of a prototype to use for the new object and a constructor function to invoke:

var Person = function(firstName, lastName) {
  this.firstName = firstName;
  this.lastName = lastName;
}
 
Person.prototype = {
  toString: function() { return this.firstName + ' ' + this.lastName; }
}

Here, we have a single Function object that is both a constructor function and an object to use as the prototype of new objects. Let’s implement a function that will create new instances from this Person object:

function newObject(func) {
  // get an Array of all the arguments except the first one
  var args = Array.prototype.slice.call(arguments, 1);
 
  // create a new object with its prototype assigned to func.prototype
  var object = Object.create(func.prototype);
 
  // invoke the constructor, passing the new object as 'this'
  // and the rest of the arguments as the arguments
  func.apply(object, args);
 
  // return the new object
  return object;
}
 
var brendan = newObject(Person, "Brendan", "Eich");
brendan.toString() // "Brendan Eich"

The new operator in JavaScript essentially does this work, providing a syntax familiar to those comfortable with traditional object oriented languages:

var mark = new Person("Mark", "Miller");
mark.toString() // "Mark Miller"

In essence, a JavaScript “class” is just a Function object that serves as a constructor plus an attached prototype object. I mentioned before that earlier versions of JavaScript did not have Object.create. Since it is so useful, people often created something like it using the new operator:

var createObject = function (o) {
  // we only want the prototype part of the `new`
  // behavior, so make an empty constructor
  function F() {}
 
  // set the function's `prototype` property to the
  // object that we want the new object's prototype
  // to be.
  F.prototype = o;
 
  // use the `new` operator. We will get a new
  // object whose prototype is o, and we will
  // invoke the empty function, which does nothing.
  return new F();
};

I really love that ECMAScript 5 and newer versions have begun to expose internal aspects of the implementation, like allowing you to directly define non-enumerable properties or define objects directly using the prototype chain.

Understanding JavaScript Function Invocation and “this”

Over the years, I’ve seen a lot of confusion about JavaScript function invocation. In particular, a lot of people have complained that the semantics of `this` in function invocations is confusing.

In my opinion, a lot of this confusion is cleared up by understanding the core function invocation primitive, and then looking at all other ways of invoking a function as sugar on top of that primitive. In fact, this is exactly how the ECMAScript spec thinks about it. In some areas, this post is a simplification of the spec, but the basic idea is the same.

The Core Primitive

First, let’s look at the core function invocation primitive, a Function’s call method[1]. The call method is relatively straight forward.

  1. Make an argument list (argList) out of parameters 1 through the end
  2. The first parameter is thisValue
  3. Invoke the function with this set to thisValue and the argList as its argument list

For example:

function hello(thing) {
  console.log(this + " says hello " + thing);
}
 
hello.call("Yehuda", "world") //=> Yehuda says hello world

As you can see, we invoked the hello method with this set to "Yehuda" and a single argument "world". This is the core primitive of JavaScript function invocation. You can think of all other function calls as desugaring to this primitive. (to “desugar” is to take a convenient syntax and describe it in terms of a more basic core primitive).

[1] In the ES5 spec, the call method is described in terms of another, more low level primitive, but it’s a very thin wrapper on top of that primitive, so I’m simplifying a bit here. See the end of this post for more information.

Simple Function Invocation

Obviously, invoking functions with call all the time would be pretty annoying. JavaScript allows us to invoke functions directly using the parens syntax (hello("world"). When we do that, the invocation desugars:

function hello(thing) {
  console.log("Hello " + thing);
}
 
// this:
hello("world")
 
// desugars to:
hello.call(window, "world");

This behavior has changed in ECMAScript 5 only when using strict mode[2]:

// this:
hello("world")
 
// desugars to:
hello.call(undefined, "world");

The short version is: a function invocation like fn(...args) is the same as fn.call(window [ES5-strict: undefined], ...args).

Note that this is also true about functions declared inline: (function() {})() is the same as (function() {}).call(window [ES5-strict: undefined).

[2] Actually, I lied a bit. The ECMAScript 5 spec says that undefined is (almost) always passed, but that the function being called should change its thisValue to the global object when not in strict mode. This allows strict mode callers to avoid breaking existing non-strict-mode libraries.

Member Functions

The next very common way to invoke a method is as a member of an object (person.hello()). In this case, the invocation desugars:

var person = {
  name: "Brendan Eich",
  hello: function(thing) {
    console.log(this + " says hello " + thing);
  }
}
 
// this:
person.hello("world")
 
// desugars to this:
person.hello.call(person, "world");

Note that it doesn't matter how the hello method becomes attached to the object in this form. Remember that we previously defined hello as a standalone function. Let's see what happens if we attach is to the object dynamically:

function hello(thing) {
  console.log(this + " says hello " + thing);
}
 
person = { name: "Brendan Eich" }
person.hello = hello;
 
person.hello("world") // still desugars to person.hello.call(person, "world")
 
hello("world") // "[object DOMWindow]world"

Notice that the function doesn't have a persistent notion of its 'this'. It is always set at call time based upon the way it was invoked by its caller.

Using Function.prototype.bind

Because it can sometimes be convenient to have a reference to a function with a persistent this value, people have historically used a simple closure trick to convert a function into one with an unchanging this:

var person = {
  name: "Brendan Eich",
  hello: function(thing) {
    console.log(this.name + " says hello " + thing);
  }
}
 
var boundHello = function(thing) { return person.hello.call(person, thing); }
 
boundHello("world");

Even though our boundHello call still desugars to boundHello.call(window, "world"), we turn right around and use our primitive call method to change the this value back to what we want it to be.

We can make this trick general-purpose with a few tweaks:

var bind = function(func, thisValue) {
  return function() {
    return func.apply(thisValue, arguments);
  }
}
 
var boundHello = bind(person.hello, person);
boundHello("world") // "Brendan Eich says hello world"

In order to understand this, you just need two more pieces of information. First, arguments is an Array-like object that represents all of the arguments passed into a function. Second, the apply method works exactly like the call primitive, except that it takes an Array-like object instead of listing the arguments out one at a time.

Our bind method simply returns a new function. When it is invoked, our new function simply invokes the original function that was passed in, setting the original value as this. It also passes through the arguments.

Because this was a somewhat common idiom, ES5 introduced a new method bind on all Function objects that implements this behavior:

var boundHello = person.hello.bind(person);
boundHello("world") // "Brendan Eich says hello world"

This is most useful when you need a raw function to pass as a callback:

var person = {
  name: "Alex Russell",
  hello: function() { console.log(this.name + " says hello world"); }
}
 
$("#some-div").click(person.hello.bind(person));
 
// when the div is clicked, "Alex Russell says hello world" is printed

This is, of course, somewhat clunky, and TC39 (the committee that works on the next version(s) of ECMAScript) continues to work on a more elegant, still-backwards-compatible solution.

On jQuery

Because jQuery makes such heavy use of anonymous callback functions, it uses the call method internally to set the this value of those callbacks to a more useful value. For instance, instead of receiving window as this in all event handlers (as you would without special intervention), jQuery invokes call on the callback with the element that set up the event handler as its first parameter.

This is extremely useful, because the default value of this in anonymous callbacks is not particularly useful, but it can give beginners to JavaScript the impression that this is, in general a strange, often mutated concept that is hard to reason about.

If you understand the basic rules for converting a sugary function call into a desugared func.call(thisValue, ...args), you should be able to navigate the not so treacherous waters of the JavaScript this value.

this-table.png

PS: I Cheated

In several places, I simplified the reality a bit from the exact wording of the specification. Probably the most important cheat is the way I called func.call a "primitive". In reality, the spec has a primitive (internally referred to as [[Call]]) that both func.call and [obj.]func() use.

However, take a look at the definition of func.call:

  1. If IsCallable(func) is false, then throw a TypeError exception.
  2. Let argList be an empty List.
  3. If this method was called with more than one argument then in left to right order starting with arg1 append each argument as the last element of argList
  4. Return the result of calling the [[Call]] internal method of func, providing thisArg as the this value and argList as the list of arguments.

As you can see, this definition is essentially a very simple JavaScript language binding to the primitive [[Call]] operation.

If you look at the definition of invoking a function, the first seven steps set up thisValue and argList, and the last step is: "Return the result of calling the [[Call]] internal method on func, providing thisValue as the this value and providing the list argList as the argument values."

It's essentially identical wording, once the argList and thisValue have been determined.

I cheated a bit in calling call a primitive, but the meaning is essentially the same as had I pulled out the spec at the beginning of this article and quoted chapter and verse.

There are also some additional cases (most notably involving with) that I didn't cover here.

Using SproutCore 2.0 with jQuery UI

One of the goals of SproutCore 2.0 is to make it trivial to integrate the tools you’re already using together with SproutCore.

One way that we do that is to make it possible to drop a SproutCore app into a small part of your existing page. Greg Moeck did a great job demonstrating this functionality on the SproutCore Blog a couple of weeks ago, and I’d definitely recommend checking it out if you’re interested in learning more about embedding SproutCore 2.0 in an existing app.

Another complementary approach is to make it easy to integrate SproutCore with existing JavaScript libraries and tools. Because SproutCore 2.0 uses templates as the primary way to build your application, it integrates well with the tools you’re already using. In this post, I’ll show you the hooks you need to integrate SproutCore with anything you want, and then show you how to do a general-purpose integration with jQuery UI.

Quick Refresh: Handlebars Templates

First of all, you can take any piece of HTML and attach it to a SproutCore view. You would typically do this in order to attach event handlers to the view or to define a function that computes how a particular value should be displayed. For instance, let’s say we have some HTML that shows a business card for a person:

<div class="card">
  <p class="name">{{salutation}} {{fullName}}</p>
  <p class="address">{{number}} {{street}}</p>
  <p class="region">{{city}}, {{state}} {{zip}}</p>
</div>

We can wrap this chunk of HTML in a SproutCore view, which will define the sources of each of the properties used in {{}}}.

<script type="text/html">
{{#view Yehuda.BusinessCard contentBinding="Yehuda.businessContent"}}
  <div class="card">
    <p class="name">{{content.salutation}} {{fullName}}</p>
    <p class="address">{{content.streetNumber}} {{content.street}}</p>
    <p class="region">{{content.city}}, {{content.state}} {{content.zip}}</p>
  </div>
{{/view}}
</script>

Wrapping the HTML snippet in a <script> tag tells SproutCore to render it using its template engine. You’d normally get the {{}} properties from a controller or other data source, but let’s define a view with the values baked in for now:

Yehuda.BusinessCard = SC.View.extend({
  // fill in the content in the template with contentBinding
  content: null,
 
  fullName: function() {
    return this.get('firstName') + ' ' + this.get('lastName');
  }.property('content.firstName', 'content.lastName')
});
 
Yehuda.businessContent = SC.Object.create({
  firstName: "Yehuda",
  lastName: "Katz",
  salutation: "Mr.",
  streetNumber: 300,
  street: "Brannan",
  city: "San Francisco",
  state: "CA",
  zip: "94107"
});

If we reload the page, we will see a static business card with my information in it. If we update my content with Yehuda.businessContent.set(key, value), SproutCore will automatically update the HTML for you. If you update any property, including firstName or lastName, which are used only as dependencies of fullName, SproutCore will automatically handle updating the HTML for you.

didCreateElement

You can also define some code to run every time an element is rendered. You can use this hook to wire up any JavaScript library to the element that SproutCore has created. For instance, let’s say we want to zebra stripe the three lines in our business card once it has been create (note: you probably would not use JS to zebra stripe in real life).

Let’s extend our view:

Yehuda.BusinessCard = SC.View.extend({
  // fill in the content in the template with contentBinding
  content: null,
 
  fullName: function() {
    return this.get('firstName') + ' ' + this.get('lastName');
  }.property('content.firstName', 'content.lastName'),
 
  didCreateElement: function() {
    this._super();
    this.$("p:even").addClass("even");
  }
});

First of all, you will want to call super, to ensure that any hooks in parent classes get called. Second, all SproutCore views have a $ function, which is the normal jQuery function, scoped to the current element. You can also call this.$() to get back a jQuery object wrapping the element itself.

You can use this hook to do anything you want, allowing you to hook in any library you want. You would use this hook a lot like you’d use a jQuery(document).ready hook, but scoped to the view being created.

SproutCore also provides a willDestroyElement hook, which you can use to tear things down that you set up when the element was created. This is relatively rare, and is mostly used when interfacing with another toolkit that requires a teardown, or to tear down custom SproutCore observers.

jQuery UI

We can use these hooks to build some basic integration with jQuery UI. I’ve written a demo that shows how to connect SproutCore bindings to jQuery options at http://sc20-jqui.strobeapp.com/. You can also check out the annotated source code.

The most important thing to look at is the JQ.Widget mixin, which can be mixed into view classes to attach jQuery UI options and events to SproutCore’s binding and event system.

You can get started with the SproutCore 2.0 developer preview by downloading the starter kit.

Until next time!

Announcing Handlebars.js

For a number of years, I’ve been about as active in client-side development (through jQuery) as I have been active in server-side development (through Merb and then Rails). Recently, I’ve released a couple of libraries (jQuery Offline and Rack::Offline), designed to make it easier to build applications that can withstand both flaky connections and a complete lack of connectivity. Of course, those libraries leveraged features in modern browsers, and have gotten a bunch of use.

For jQuery Offline, the basic strategy is that instead of requesting a new JSON object before you can show anything on the page, you first try to retrieve that data from cache, show some stale content. In parallel, you’d download the new content (if possible), and update the display with the content in question. You’ve probably seen this pattern at work in applications like the official Twitter application for the iPhone, which shows stale content when you first boot the app, and downloads new content if you have Internet connectivity in the background.

To take the JSON and convert it into some HTML for the page, I recommended jquery-tmpl or mustache.js. Both of these libraries allow you to take a JSON object and convert it on the fly into HTML, which you can insert into the page. Both libraries are still very good, and I could easily imagine problem-free uses for both of them.

Of the two libraries, I really liked Mustache’s approach. Mustache envisions a template as something that is completely code-free, and populated via a JavaScript Object. This Object can contain normal values, or functions which are used as helpers.

<script type="text/x-mustache-template" name="posts">
{{#posts}}
<h1><a href="{{url}}">{{title}}</a></h1>
 
{{#div}}
{{body}}
{{/div}}
{{/post}}
</script>

You’d then populate this template with a JavaScript object:

var view = {posts: [
  { id: 1,
    url: "/posts/1",
    title: "Hello world!",
    div: function(text, render) { return "<div class='post entry'>" + render(text) + "</div>" },
    post: "Hello world. This is my first post"
  },
  { id: 2,
    url: "/posts/2",
    title: "Goodbye world!",
    div: function(text, render) { return "<div class='post entry'>" + render(text) + "</div>" },
    post: "Goodbye world. Sadly, this is going to be my last post"
  }
]};
 
var template = $("script[name=posts]").html();
var html     = Mustache.to_html(template, view)

The basic idea here is that the template itself is completely free of code, and just uses keys from the JavaScript Object to convert it into HTML. The JavaScript Object can have some functionality, and even code, to help the template along. I really like the idea of limiting the template to referencing an element of the JavaScript object, and completely firewalling any kind of code to the view object. Among other things, this allows a single template to run in both Ruby on the server-side and JavaScript on the client side.

When using Mustache, I ran into a few issues. One minor issue was that the base implementation of mustache.js was essentially interpreted, rather than compiled. This means that the implementation is slower than you might expect of a template language, especially one as limited as Mustache.

More concerningly, the base Mustache syntax was too limited for a lot of things I wanted to use it for. I wasn’t immediately sure if it was just me, so I asked a bunch of people about their opinions of Mustache. One of the people who replied was Alan Johnson, who had been using Mustache a lot and volunteered his thoughts. Without much prompting, Alan’s concerns about the limitations of Mustache (in both our cases, born from real attempts to use it) closely mirrored mine. My thinking validated for the moment, I got to work trying to extend Mustache to get around these limitations.

Before I continue, here are some of the more important problems I had:

  • Function values (like div above) needed to be located in the same place in the hierarchy as object being used as a context. This made providing a set of global helpers (like a framework would want to do) quite difficult.
  • Even if the syntax was modified to make function helpers global, the functions would not have access to the current context, so they would be of limited value (imagine trying to change the url key above into a helper)
  • I commonly wanted access to a key in a parent scope when working in a child scope. For instance, if a post had many comments, I would sometimes want access to the post’s title (or other details) while rendering the child

There were other small irritations, but these were definitely the major ones. A lot of the people I spoke to solved these problems by rewriting the JavaScript Object before passing it into the Mustache system. Some of my initial thinking, in fact, revolved around making that sort of rewriting easier. In the end, though, I wanted to extend the Mustache syntax with a few new features that would make it behave more like what I expected.

From the beginning, I wanted my extensions to be a strict superset of the official Mustache manual, which is pretty clear in some areas and somewhat more vague in other areas. I really like the Mustache syntax and philosophy, and wanted to make sure whatever I came up with would not diverge philosophically from what I enjoyed about Mustache itself.

Compiled, Not Interpreted

The first step was to make the Mustache syntax compiled, not interpreted. There is another project (called Mu) which is a compiled variant of Mustache, but it was explicitly built for Node.js, and is built around server-side JavaScript, rather than client-side JavaScript. I also wanted to build it with some of the extensions I wanted to add in mind from the beginning.

The parser/compiler is a lot like the ERB compiler, and has a lot of what you’d expect from a parser/compiler (getChar, peek, addText, addExpression, etc.). I designed it from the beginning to be extended, taking some of the architecture from Erubis, which separates out how different kinds of tags are added to the source it is building up. We’ll probably have to make a few tweaks before extensions can be easily dropped in, but I already have a couple in mind that will help flesh out the API.

Extended Paths

The first extension to Mustache is “extended paths”. In Mustache, paths are always relative to the current context (which can change with iteration), and are always a single key. For instance, the url path from the above example references the url key in the current post that Mustache is iterating over.

Extended paths allow you to go deeper into the Object (byline/name) or up the hierarchy (../title). This doesn’t diverge from the core Mustache philosophy of avoiding code in templates, but it does allow templates to more flexibly reference other parts of the Object, apart from the context they are in the process of rendering. A lot of people implement this feature by rewriting the Object before passing it into the template–Handlebars allows that rewriting to happen on the fly.

Global Lambdas with Access to the Current Context

Mustache allows the Object to contain lambdas, but they must appear in the same context as the rest of the keys being rendered. If iterating over a list of projects, for instance, each project must have a key referencing the lambda, even if it’s always the same. Handlebars allows you to provide a global Object of lambdas, which it will look in when it does not find a key in the current context. This allows you to create helpers to use throughout your Template. Again, people implemented this feature by rewriting the Object, and Handlebars is just making it possible to achieve the same goal without that rewriting.

After adding global lambdas, those helpers might want to do different things based on the context. Handlebars invokes its helper functions with a this that reflects the current context. We can rewrite the above example as:

var view = {posts: [
  { id: 1,
    title: "Hello world!",
    div: function(text, render) { return "<div class='post entry'>" + render(text) + "</div>" },
    post: "Hello world. This is my first post"
  },
  { id: 2,
    title: "Goodbye world!",
    div: function(text, render) { return "<div class='post entry'>" + render(text) + "</div>" },
    post: "Goodbye world. Sadly, this is going to be my last post"
  }
]};
 
var helpers = {
  url: function() {
    return "/posts/" + this.id;
  }
}

Of course, this example is pretty simple, and you could have just done {{title}}, but this illustrates that helpers can use the current context to control the output.

Helpers Take a Context

Helpers could be made even more generic if they could take a context (referenced as a path) as an argument. As I’ve shown above, helpers are always executed with their current rendering context as this, but it could be convenient to pass a different context, referenced by its path, to the function.

{{#projects}}
  <p>Back: {{url ..}}</p>
 
  <div>
    {{body}}
  </div>
{{/project}}

In this example, we want each project to include a link back to the parent project. By passing in a context (in this case, the object referenced by the extended path ..) we can make our helpers even more generic.

var helpers = {
  url: function(context) {
    return "/" + context.type + "/" + context.id;
  }
}

Block Helpers

The Mustache manual is a bit light on how lambda functions should work, referencing a render method that it doesn’t pass into the function. I wanted block helpers to take a function that was essentially identical to the compiled function for the entire template, and would be populated in the same way a regular template gets populated. A good example use for this type of helper is a custom list.

<script type="text/x-handlebars-template" name="items">
{{#list items}}
<h1>{{title}}</h1>
<div>{{body}}</div>
{{/list}}
</script>
var view = { items:
  [
    { title: "Hello world",         body: "This is my first post" },
    { title: "Goodbye cruel world", body: "This will be my last post ;(" }
  ]
}
 
var helpers = {
  list: function(context, fn) {
    var buffer = "<ul class='list'>\n";
    for(var i=0, j=context.length; i < j; i++) {
      buffer.push( "<li>" + fn(context[i]) + "</li>\n" )
    }
    buffer.push("</ul>");
  }
}
 
var source   = $("script[name=items]").html();
var template = Handlebars.compile(source);
 
template(view, helpers)

This will output:

<ul class='list'>
<li><h1>Hello world</h1>
<div>This is my first post</div></li>
<li><h1>Goodbye world</h1>
<div>This will be my last post ;(</div></li>
</ul>

As you can see, block helpers can now invoke the sub-template many times with different contexts. And block helpers, like regular helpers, can receive a context, expressed by an extended path, as a parameter.

Iteration and Booleans Are Just Block Helpers

One thing that bugged me was that iteration and booleans were special-cased in the normal Mustache specification. I didn’t want to change the behavior of iteration or booleans, but I did want to make them behave the same as regular block helpers. By making block helpers more powerful, I was able to achieve this with one small additional change.

In particular, rather than hardcode the behavior of iteration and booleans, I created a new feature allowing a particular function to be used when a block helper could not be found. This way, every use of {{#foo}} was a block helper, but iteration was just invoking the special helperMissing block helper. In both this case and the case of conditionals, the missing helper was treated as a context to be looked up as a path.

The helperMissing function itself is quite simple:

Handlebars.helperMissing = function(object, fn) {
  var ret = "";
 
  if(object === true) {
    return fn(this);
  } else if(object === false) {
    return "";
  } else if(toString.call(object) === "[object Array]") {
    for(var i=0, j=object.length; i < j; i++) {
      ret = ret + fn(object[i]);
    }
    return ret;
  } else {
    return fn(object);
  }
};

First, if the object referenced is true, we invoke the template block, using the current context. If the object referenced is false, we return an empty String.

If the object referenced is an Array, we iterate over the elements of the Array, invoking the template block once in the context of each element of the Array. This looks a lot like the items block helper we created above.

Finally, if the object referenced is anything else, we invoke the template block in the context of that object. Because we made block helpers more powerful, it is easy to implement the hardcoded boolean, iteration and context-switching behavior of the Mustache spec in terms of the block helpers. Note that if you wanted to change this behavior, you would just need to override Handlebars.helperMissing with a new function of your own design.

Better Inverted Sections

Finally, Mustache allows the specification of inverted sections, which behave like the reverse of a block. For instance, where {{#boolean}} will run the block if boolean is true (or a non-falsy value), and skip it if boolean is false, {{^boolean}} will run the block if boolean is false, and skip it if boolean is true (or non-falsy value).

A very common usage of inverted sections is to behave like if/else blocks:

{{#project}}
<h1>{{name}}</h1>
<div>{{body}}</div>
{{/project}}
{{^project}}
<h1>No projects</h1>
{{/project}}

Because of how common this is, Handlebars has a shortened syntax for this case:

{{#project}}
<h1>{{name}}</h1>
<div>{{body}}</div>
{{^}}
<h1>No Projects</h1>
{{/project}}

Additionally, you can specify the inverse form of a custom block helper by assigning a not property on the helper function.

{{#list projects}}
<h1>{{name}}</h1>
<div>{{body}}</div>
{{^}}
<h1>No Projects</h1>
{{/project}}
var helpers = {
  list: function(items, fn) {
    if(!items.length) return "";
 
    var buffer = "<ul>";
    for(var i=0, j=items.length; i < j; i++) {
      buffer += fn(items[i]);
    }
    buffer += "</ul>"
    return buffer;
  }
}
 
helpers.list.not = function(items, fn) {
  if(!items.length) { return fn(); }
}

Wrapping Up

Long story short: the goal of handlebars.js is to create a syntax-compatible superset of Mustache that solves a lot of problems I’ve had when working with Mustache (especially as part of a larger library).

I want to personally thank Alan Johnson for contacting me early and fleshing out my initial prototype into a workable, releasable library. His experience with Mustache, as well as his willingness to spend time on the nitty gritty details really made the difference. Thanks!

As always, please submit bug reports to the GitHub Issues tracker associated with the project. I really love the new GitHub pull request system, so please submit patches using it.

The Irony of the iPad: A GREAT Day for Open Technologies

With the announcement of the iPad, the usual suspects have come out decrying a closed, proprietary, fully locked down system.

For instance, a story on the top of Hacker News today says:

This is what I asked in January 2007 on this site, shortly after the original iPhone was launched:

“1. Will Apple lock down the iPhone, blocking Flash, Java, custom widgets, and open development from its new platform?

2. Could Apple’s multi-touch patents actually stifle growth of new, interactive displays?”

Unfortunately, that turned out to prescient

And the FSF is out there calling this an unprecedented march of DRM:

With new tablet device, Apple’s Steve Jobs pushes unprecedented extension of DRM to a new class of general purpose computers

It’s a fair initial reaction. Apple didn’t build a general-purpose computer as its next entry into the market. Instead, they built a heavily proprietary, locked down device. In order to install an application onto the device, Apple must approve the application.

I don’t need to address the merits of the argument against how Apple handles native applications, because it’s irrelevant. A much, much larger force is at work here.

With the iPad, Apple has created two platforms. First, they have produced a heavily proprietary, native platform that requires Apple approval and has significant Apple restrictions. But ironically, with their heavy focus on improving the quality of Safari and the HTML standard, they have shipped the iPad with a platform based on open, unencumbered technologies.

If you haven’t been paying attention, over the past couple of years, the web platform has gotten offline APIs, improved caching support, local storage (on Safari, that includes an on-device SQLite database accessible through JavaScript), CSS-based animations, and custom, downloadable fonts. Mobile Safari has support for gestures, Geolocation, and hardware-accelerated graphics.

Additionally, Apple has remained at the forefront of these technologies, literally building some of them for mobile devices (hardware-accelerated animations were built for the iPhone, and by extension, the iPad). The Open Source Webkit project has remained extremely active, and in fact, has only accelerated progress since Apple first released its Native SDK, so Apple’s “locked down” strategy has a very carefully carved out intentional exception.

Apple even makes it easy to take a web app and put it on the home screen amongst normal apps. When you do this, the iPhone downloads all the assets in the HTML5 cache manifest to make the work better as an offline app. This is how I use Gmail on my iPhone (because Google knows what’s going on in this space, they leverage new tech in Safari quite well). When Apple rejected Google Voice, Google immediately built a Safari version of the app. The way they tell it:

Already, Google says it is readying a replacement for the Google Voice app that will offer exactly the same features as the rejected app—except that it will take the form of a specialized, iPhone-shaped Web page

Ironically, despite claims that not allowing Flash or Java represent a victory for proprietary technologies and a loss for open technologies, they represent quite the opposite. By restricting the web platform on the iPhone and iPad to open, patent-free, technologies, Apple has created a highly desirable market for pure-HTML5 apps. This is, frankly, a win for supporters of open technologies.

jQuery 1.4 and Malformed JSON

Today, we released jQuery 1.4, a mostly backward compatible release with a few minor quirks. One of these quirks is that jQuery now uses the native JSON parser where available, rejecting malformed JSON.

This probably sounds fine, except that by malformed JSON, we mostly mean:

{foo: "bar"}

Because this is valid JavaScript, people have long assumed it was valid JSON, and a lot of hand-coded JSON is written like this. Thankfully, JSON automatically generated by Rails does not have this problem, so most of you guys should not have an issue here.

If you have server-side JSON that can’t be quickly fixed in the transition to jQuery 1.4, you can use the compatibility plugin, which tries to patch a number of small incompatibilities, or you can use this workaround:

$.ajax({url: "/url", 
  dataType: "json",
  success: function(json) {
    // do something with json
  }
});
 
// becomes
 
$.ajax({url: "/url",
  dataType: "text",
  success: function(text) {
    json = eval("(" + text + ")");
    // do something with JSON
  }
});

Essentially, this is what jQuery used to do to evaluate JSON, so with these small changes you can get the old behavior.

Again, though, you should probably be modifying your busted server-side code to stop sending malformed JSON!

Files from my talk at @media

I have uploaded the files that I will use for my demonstration at @media. The slides will come later today after my talk. The easiest way to browse the files are via github. You can either browse the files online, get them via git, or download a tarball of the entire thing.

If you’re at @media, feel free to download the files and follow along.

RailsConf Europe Slides

Hey guys!

My RailsConf Europe slides are up :)

http://yehudakatz.com/wp-content/uploads/2008/09/rce-jquery.pdf

If you were at my talk, please log into your account at O’reilly and rate it. Thanks!

method_missing in JavaScript?

I’ve posted previously about JavaScript 2 (also known as ECMAScript 4). Since I last discussed this topic, the debate over whether to continue with ES4, which extended JavaScript with type annotations and classes, got significantly less cooperative, and apparently came to a head at the meeting of TC99, the committee responsible for the future of JavaScript.

In summary, it seems that the two groups agreed to focus cooperative effort of ECMAScript 3.1, a modest improvement of the current JavaScript that includes some new features like the ability to mark properties as non-enumerable and freezing objects (which can be used to implement classes).

After that is complete, the two groups have agreed to work on a new version of JavaScript dubbed “Harmony”, which will be a more modest evolution of the current JavaScript, minus some of the more ambitious features like namespaces and packages. Other features, like classes, will likely be implemented in terms of new features in ES3.1 like freeze().

Which brings me to the title of my post. Now that it seems as though ES3.1 will be embraced by all the browser vendors moving forward (and will likely be the implemented iteration of JS for at least a little while), I wondered what sorts of features might still make it into ES3.1 before the spec was closed.

Specifically, I’ve long been interested in trying to get method_missing into JavaScript, and in fact, it is already available in Firefox/Spidermonkey as __noSuchMethod__. In fact, Johnson makes use of noSuchMethod in its Ruby proxy.

Half-wishing, half-hoping, I threw the idea of codifying noSuchMethod into the discussion on the ES discussion group. As expected, Brendan Eich replied that it was probably too late to add a feature like this into ES3.1 at this time.

But then, to my surprise, Bill Edney and Scott Shattuck, who were responsible for getting noSuchMethod into Spidermonkey, jumped into the fray urging its inclusion into ES3.1. And Justin Meyer, of JavaScriptMVC, posted to their blog also urging its inclusion.

Anyone who’s messed with Ruby for more than just a simple Rails app understands the power of method_missing, and getting it into ES3.1, which would mean it would make it into future versions of Firefox, Safari, and IE, would be sweet indeed.

Archives

Categories

Meta