A couple of months ago, I read The End of Overeating, which got me started on a series of books about food. I worked my way through In Defense of Food, The Omnivore’s Dilemma, and Animal, Vegetable, Miracle.

After reading through these books and doing a bunch of auxiliary research, I came to the fairly disturbing conclusion that the food we eat from the grocery is sorely lacking in required nutrients. That’s especially true for packaged, processed foods, but even the fruits and vegetables purchased in a typical produce section are lacking.

Studies show that fruits and vegetables grown without pesticides or herbicides have significantly higher levels of certain vitamins and antioxidants. And simple replacements of the missing compounds assumes that we have a full understanding of what’s missing. We don’t.

Additionally, breeding fruits and vegetables for high yield, uniform appearance, and long travel distance necessarily reduces other more important factors, like taste and nutritional content. Like I said, studies have found this to be true, but in retrospect, it’s fairly self-evident. Evolution is a process of competition among zero-sum ends. If yield and long-distance travel win out, something else loses out.

With that introduction, over the past few months, I’ve slowly started working toward cooking virtually all of my own food. In the past few weeks, that has expanded to include breads and sauces. In short, the rule I try to follow is: “Only purchase items with a single ingredient in their ingredient list.” And at a very minimum, a rule I got from Michael Pollan’s books: “Only purchase items with a few ingredients, all of which are understandable.”

It probably sounds like I’ve absorbed too much of San Francisco’s culture, but I have to tell you, the quality of the food I’ve been eating has increased dramatically. For the most part, food tastes better, and even when it doesn’t, I really enjoy putting together meals.

In order to make this happen, I’ve purchased a few pieces of equipment. First of all, I purchased a bread-maker. It cost only $100, and I’ve already made two loaves of pretty good tasting whole-wheat bread.

I also purchased a rice maker, which makes cooking brown rice myself actually possible. Trying to do it on a stove yourself is basically impossible (try Googling instructions on cooking brown rice).

I dramatically increased my consumption of organic fruits and vegetables, much of it locally grown. After a month or so, I can absolutely confirm that the quality and taste of the fruits and vegetables is significantly higher, and I’ve straight-up stopped worrying about the small differences in fat in things like whole milk and skim milk, since those items are mostly garnishes on fruits and vegetables. As a result, I’ve been able to focus on the taste of my ingredients, instead of trying to squeeze a few more grams of fat out of a meal, and I’ve still been able to lose a bunch of weight since I started (again, in retrospect, it’s not all that surprising that eating a ton more fruits and vegetables, regardless of what else is in the diet, would result in weight loss).

From The End of Overeating, I also completely cut out snacks (snacks are actually a relatively new Western invention), focusing heavily on three meals a day, which increases the quality and enjoyment of actually eating food that I prepare carefully and well.

Finally, I joined a local CSA, which delivers a weekly box of vegetables. I got my first shipment today, which brought me 1.5 pounds of yellow peaches, 1.5 pounds black plums, 6 oz. blueberries, 1 pund summer squash, 1 bunch chard, 1/2 pund gypsey peppers, 1/2 pound lipstick peppers, 1.5 pounds heirloom tomatoes, 1 bunch greenleaf lettuce, 1 bunch red beets, 1 bunch nantes carrots, and 1 pound red onion. All from local farms, all organic, and all for just $30.

In the past, when I heard people talking about stuff like this, they sounded like kooky hippies, so I suspect that’s how I sound to people as well. But if eating tasty, nutritious food that I prepare myself, losing weight and saving money is a kooky hippie thing to do, I’ll take kookie hippie any day.

While we’ve been working on ActionController::Base, there were a few tests for fixes in Rails 2.3 that were mostly hacked in and we were waiting for a cleaner underlying implementation to build them on top of. Today, we finally got all the pending tests passing!

There were basically two categories of pending tests:

Layout Selection

Rails 2 had a feature called :exempt_from_layout, which allowed users to specify that a particular kind of layout (defaulting to RJS) should be exempt from layout. The reason for this feature was that people were noticing that their RJS layouts were being wrapped in their HTML layouts.

When I first saw this feature, I had a simple question: “Why were RJS layouts being wrapped in HTML templates in the first place?” And also, if you want an RJS layout (application.js.erb), why shouldn’t you be allowed to.

As it turns out, the reason for this was that any number of parts of Rails render templates, and the layout lookup is completely separate. So when it came time to look up a layout, Rails didn’t realize that it had just rendered an RJS template (as far as it was concerned, since the available formats allow for HTML and JS, either would do).

The solution: Have template and layout lookup go through a deterministic process, and limit layout lookup to the mime type for the template that was actually rendered (not templates that might have been rendered).

The upshot: exempt_from_layout isn’t needed. If you don’t want a layout around your RJS templates, don’t make an application.js.*. If you do, do. Rails will do the right thing.

An aside: the hardest part of Rails layouts involves when to raise an exception for a missing layout. Since Rails allows layouts to be missing in certain circumstances, making sure an exception is raised in other circumstances is quite tricky. In master, we raise an exception if you explicitly provided a layout (layout "foo"), and none exist for any MIME type. Implicit layouts or explicit layouts that exist for another MIME are permitted to be absent. The exception to that rule is render :layout => true, which converts an implicit layout to a required, explicit one.

RJS and HTML

Rails 2 has a bunch of hardcoded rules that allow RJS templates to render HTML. This allows page[:foo].replace_html :partial => "some_partial" to render some_partial.html.erb.

Effectively, when you got into an RJS template, the acceptable formats list was hardcoded to [:html]. Again, this blocks the use of RJS partials, and if you supply only an RJS partial, it will not be used (a missing template error would be raised).

When we thought about it, we realized that what is desired is to look for templates matching the mime type of the current template first, and if such a template could not be found, to allow templates matching any mime (with HTML leading the list). If you’re in an RJS template and you call render :partial => "foo", and only a foo.xml.erb exists, we can assume that you mean to render that template.

This handles all of the cases supported by Rails 2, with one small change. If you have both a js and html template, the js template will win inside of RJS. If you didn’t want the js template to win, why did you create it?

That rule now applies to any template. Partials matching the existing mime will be rendered if they exist, but any other mime will work fine as a fallback. So no special rules required for RJS anymore. The same rules apply for other templates (:file etc.) rendered from the context of another template.

One last thing. If you do page[:customer].replace_html :partial => "world", an html template will be required. When we noticed that we could separate out the rules for replace_html from other partials, we realized that we could hardcode more restrictive rules for that specific API than the general API.

And with that, the remaining pending tests pass. I really enjoy being able to solve problems that required hacks in the past by reorganizing the code to produce conceptually nicer solutions.

There’s been a fair bit of discussion around RubyGems lately, and some suggestions about what the core problems with RubyGems are.

People have the general sense that there’s something wrong with dependencies, and that it might have something to do with multiple versions being installed in one repository. It also seems (to people) that having require do magical things is Bad(tm). And in general, people like knowing exactly what versions of things are being loaded.

To some degree, all of these concerns are valid, and led to the rather hackish solution that we distributed with Merb called merb.thor. What we did:

• Created a manifest for your application that would describe the gems and versions you wanted to use. That same manifest was used at runtime to load those gems.
• Create a virtual environment just for your application, with the one-version-per-environment rule. This meant that it was always possible to see what versions and gems were being used.
• Make it reasonably easy to update the local environment when the manifest changes. Make such changes *not* require knowledge of the dependencies and versions of either the old or new gems.

What we did not do:

• Put all the gems in a single directory, so normal Ruby require would work.

At first glance, this seems like a very good idea. Instead of relying on magical runtime load-path manipulation, just take, for instance, the merb-core gem, and stick it in a top-level. Then add that top-level to the load path and you don’t need Rubygems at runtime.

The problem with this fabulous idea is that there isn’t a consistent way that people use Rubygems. Consider the following scenario:

A gem called “bad-behavior” that has a lib loadpath, but puts server.rb, initializer.rb, and omg.rb at the top-level. In omg.rb, the gem does Dir["#{File.dirname(__FILE__)}/*"].each {|f| require f }. This works fine when the gem actually owns the entire directory. But if you drop the gem into a larger file structure (similar to how other package managers handle the problem), its top-level is now everyone else’s top-level.

Another scenario: A gem called rack-silliness that puts its files in rack/*, and then calls Dir["#{File.dirname(__FILE__)}/*"].each {|f| require f } from rack/silliness.rb. Again, this works fine if the gem owns the entire directory, but if multiple gems put things in rack/*, moving everything to a shared structure will fail.

With all that said, if we *could* use a shared structure, things would automatically fall into place. We wouldn’t need rubygems at runtime. It would be easy to have separate environments with the one-version rule. It would be easy to have local environments. *All within the existing Rubygems structure*.

The solution I promised

So how do we solve this problem? We need to agree to deprecate everything but the following structure for Rubygems:

Given a gem foo, there should be a foo.rb at the top-level, and optionally, a foo directory underneath. No other files or directories are allowed

Update:What I meant here was lib/foo.rb and lib/foo/…, which will be the directory that gets added to the load path. As a result, the vast majority of existing gems would not need to change.

Other solutions that work with Rubygems but use a single shared directory structure *assume* well-behaved gems only. If we could enforce well-behaved gems, we would both have an excellent solution in Rubygems proper, and make it easier for people to build additional solutions and plugins around the gem format.

So here’s my proposal: For the next version of Rubygems, print a warning if installing a gem that does not comply. Over the next few months, get the few existing gem authors who have non-complying gems to release new versions that comply.

At the same time, I will release a gem plugin that provides virtual environments and local environments for Rubygems (I have already been working on this). It will support the one-version rule, named virtual environments, a gem manifest for applications, and gem resolution (thanks to the hard work by Tim Carey-Smith on gem_resolver).

In the interim, we have a slightly clunky solution that will work well. Instead of putting all gems into a single load-path and using that, we leave the current structure (each gem has its own space). Then, when a gem is installed into an environment, we preresolve all load-paths, and keep a list of them. When you switch into an environment, we add those load-paths to the default set of Ruby load-paths, which will behave exactly the same, but still support misbehaving gems.

In the long-term, all gems will be able to live side-by-side in a single load-path, which will allow us to create a cleaner version of the virtual environments (and will improve startup times, especially on JRuby and Google App Engine, but won’t have any user-facing implications).

So, are we up for finally getting our gem packaging format under control?

P.S. I am aware that rip was just announced, and is attempting to do a lot of the same things. This blog post has been a long time coming (the ideas were hatched a year ago, and many are available today as part of Merb). What I’d like to do here is take the good ideas that exist in Merb, rip, and the Python community and make them native to Rubygems, addressing the problems I outlined above that are inherent to the transition. It’s perfectly fine for rip to simply require well-formed gems, but a solution that gets us from here to there as a community is important.