On Rails Testing

One of the things that has both pleasantly surprised and frustrated me over the past six months is the state of Rails' internal tests. While the tests can sometimes cover the most obscure, important bugs, they can sometimes be heavily mock-based or very coupled to internal implementation.

In large part, this is because of the requirement that patches come with tests, so the test suite has accumulated a lot of knowledge over time, but isn't always the clearest when it comes time for a major refactor. Just to be clear, without the test suite, I don't think the work we've been doing would have been possible, so I'm not complaining much.

However, I recently became aware that Sam Ruby, one of the authors of Agile Web Development on Rails, has released a test suite that tests each step in the depot application, as well as each of the advanced chapters.

Last week, Carl and I started digging into the Rails initializer, and the tests in the initializer (railties) are more mock-based and less reliable than the tests in ActionPack (which we've been working with so far). They're pretty reasonable unit tests for individual components, but getting all of the tests to pass did not result in an (even close) bootable Rails app. Thanks to Sam's tests, we were able to work through getting a booting Rails app by the end of last week.

Over the past week or two, Carl and I have been running the tests a few times a day, and while they definitely take a while to get through, they've added a new sense of confidence to the work we're doing.