Rails 3 Beta is Out – A Retrospective

The Rails team has finally released the Rails 3 beta, after more than a year since the Rails and Merb teams started working on this release. You can read all about it at the official Rails blog, but I figured I’d take the opportunity to share my take on the release.

First of all, you’re probably sick of hearing this, but we’ve done far, far more than we ever expected. A lot of that happened in the last few weeks.

One of the things that most surprised and impressed me is the Rails core team’s (and especially DHH’s) attention to detail and the experience of the beta release. For weeks, we’ve been “this close” to releasing, but the experience of starting up a new Rails app or upgrading from a Rails 2.3 app still felt too unpleasant. In this kind of situation, it’s tempting to say “it’s just a beta—people who use beta software know what they’re getting,” but that would have been a major cop-out. For many people on the leading edge, a poor beta experience will shape their perception of the product as a whole. So we waited a bit, but now we’re finally here.

In addition to a surprising (in a good way) amount of polish, we touched nearly every part of Rails: I’ll walk you through the top-level blocks.

Dependencies

Rails gets a bunch of new dependencies, stepping out of the “virtually zero dependencies” box it has lived in since the beginning. In order to achieve this, we wrote a brand new dependency management system (called Bundler) from scratch, remedying many of the problems with Rubygems that have made using a number of dependencies difficult (and which we encountered in Merb when taking a more-dependencies approach). Taking another step into the future, we built Bundler itself as a standalone component, so you can use it for Rails, Merb, Sinatra, or even a plain old Ruby script.

And even though the Rails happy path uses Bundler to manage dependencies, that happy path is limited to the generated code only. The Rails library itself does not rely on Rubygems, Bundler, or any package manager. This makes using a different package manager, like Debian or rip, as simple as changing a few lines of the generated code.

Instrumentation

A number of companies and projects have sprung up over the years attempting to provide useful information about the runtime status of a Rails process. In order to facilitate this work going forward, we’ve baked instrumentation into Rails itself. Instead of having to monkey-patch an appropriate method to learn about which templates Rails renders, for instance, simply listen for an event. While Rails has never officially supported the monkey-patching approach, we will commit to supporting instrumentation events in the same way as we support any other public APIs, making them future-proof stable ways to learn about what’s happening inside of Rails.

Better yet, Rails’ logging system uses these very events to emit information into the log!

Every Framework is an Extension

In what is possibly the biggest conceptual shift in Rails 3, we’ve exposed an extension API that allowed us to make ActiveRecord, ActionController, and ActionMailer simple Rails extensions. In other words, the DataMapper ORM can take a look at the part of ActiveRecord that implements its Rails extension, implement similar features, and know for sure that it’ll work. Additionally, users can completely remove the ActiveRecord gem, and be rid of the generators, rake tasks and other peripheral elements. An example: if DataMapper implements a db:test:prepare rake task, a Rails developer can replace ActiveRecord with DataMapper, and the test:units rake task will use DataMapper’s new tasks.

In short, we have completely removed references to other frameworks from Railties itself, making the frameworks of Rails true extensions to Rails itself. This makes it possible to replace any part of Rails without having to worry about entanglements with Railties.

Rack

Rails 2.3 took a very firm stance in favor of Rack. Rails 3 completes the story. In Rails 3, the Application itself, the router, and controller actions themselves are all Rack applications. Rails ships with a standard config.ru, and can boot using the rackup tool that ships with Rack, or with Passenger or Glassfish in standard Rack mode. In fact, any Rails host or server that supports the Rack protocol can run a Rails 3 application unaltered.

Because so many parts of Rails are Rack, you can also insert standard Rack middleware in config.ru, in your Rails application, in the router, or even in a controller’s actions. Rails itself ships with a number of standard middlewares, for exception handling, cookies, parameter parsing, flash notifications, sessions and a bunch more. You can use any of these middlewares in standard Rack applications (without Rails).

ActionController

ActionController got a major overhaul, dividing up a monolithic component into multiple pieces. ###ActionDispatch You can think of ActionDispatch as rack-ext. Essentially, we packaged up all of the parts of ActionController involved in the Rack dispatching process into a single reusable component. This includes ActionDispatch::Request, ActionDispatch::Response, our mime type handling, a bunch of middleware, and integration testing support.

It also includes a wrapper around rack-mount a new powerful routing library. Josh Peek, a member of the Rails Core Team, wrote rack-mount as a standalone component, so that it could be used in standard Rack projects without any Rails dependencies. By default, it simply dispatches inbound requests to Rack endpoints.

Because Rails 3 controller actions are Rack endpoints, Josh could easily wire up a thin shim around rack-mount to dispatch inbound requests to Rails controller actions. But because we built the Rails router on top of rack-mount, Rails applications can route inbound requests to any Rack application, like Sinatra or Cloudkit. ###AbstractController The disparity between ActionController and ActionMailer has long been a personal pet peeve of mine. For Rails 3, we refactored ActionController and ActionMailer to inherit from a common superclass: AbstractController. AbstractController includes the basic concept of controllers, rendering, layouts, helpers, and callbacks, but nothing about HTTP or mail delivery. This means that ActionMailer can no longer drift away from ActionController, leaving us with two distant cousin APIs. ###ActionController ActionController got a big overhaul, completely eliminating alias_method_chain in favor of a module-based system that extension author can more easily extend. For instance, we exposed a Renderer interface, allowing extension authors to easily add render :pdf or render :csv without having to override the render method in brittle ways.

While the API did not change much in general, we did add a few new features, such as the ability to add middleware directly to a controller’s actions or the ability to wrap up common idioms more cleanly using responders. Additionally, we improved the overall architecture to reduce the need for certain hacks.

For instance, ActionController will now look up the layout for an action using the details (such as format and locale) that its template used. In Rails 2.3, an RJS template could end up using an HTML layout, requiring us to exempt RJS templates from layouts altogether. However, this was a band-aid (what if you wanted an RJS layout?) and the improved architecture eliminates the possibility for other, similar problems. ###ActionView ActionView got a significant overhaul, improving its internal architecture, increasing performance, adding pervasive XSS protection, and making all JavaScript helpers unobtrusive.

In terms of architecture, Rails 3 defines a clear boundary between ActionController and ActionView. Now, ActionView exposes a single API entry point for rendering templates and partials, making the code path easier to understand, eliminating a number of latent (but unusual) bugs, and making it easier to reason about caching lookups for performance.

When doing a performance audit of Rails applications, ActiveRecord and ActionView (specifically, template rendering) pop up as major costs. In particular, ActionView has had a fair bit of overhead that’s impossible to avoid without throwing certain features out the window. For instance, partials and especially collections of partials had enough overhead to become noticeable on profiles of real applications.

In Rails 3, we optimized both the template lookup logic (which should now take up extremely small amounts of time) and the logic inside the loop of partial collections. In particular, partial collections should often be much faster in Rails 3 than a manual loop, and not much slower than manually inlining and unrolling the loop. Finally, the improved architecture has made additional optimizations possible, so expect further performance improvements in Rails 3.1.

We also switched to Erubis, a fully compatible ERB implementation with much better extensibility characteristics. In particular, it allows us to easily hook in the code that it generates for any ERB construct. This will allow compile-time performance optimizations in the future and enables the next feature: pervasive XSS protection.

Rails has always shipped with a simple way to escape any user-generated content. Unfortunately, this required constant vigilance. A number of highly visible, successful attacks against websites on a number of platforms taught the world that successful mitigation of this threat requires treating all Strings as suspicious, and allowing developers to mark Strings as safe. Rails 3 escapes all Strings, allowing developers to convert Strings into safe Strings easily. We have also modified all of our internal helpers to mark Strings that we create as safe, while not modifying Strings passed in by the application developer.

Finally, we have overhauled all of the JavaScript helpers to emit plain HTML markup instead of JavaScript. As part of the process, we have created drivers for jQuery and Prototype that bind events to the appropriately marked up elements in the DOM. This means that Rails can now support any JavaScript library as a first-class citizen. We will support and maintain the markup and JavaScript developers can develop pure-JavaScript libraries to make use of the markup (without ever having to touch Ruby). If other frameworks emit the same markup, the same JavaScript libraries will work with their helpers.

ActiveModel

Over the years, people have created a number of ActiveRecord lookalike libraries for things like MongoDB or CouchDB. In Rails 3, we have extracted elements of ActiveRecord unrelated to RDBMS persistence, such as validations, callbacks, dirty tracking, and serialization into a separate library. Other libraries, or even plain old Ruby code, can use these components on an à la carte basis, choosing to look as much or as little like ActiveRecord as they wish.

Additionally, ActiveModel defines an API that any object can implement in order to work seamlessly with ActionPack. It provides drop-in implementations of the API, but Ruby libraries that wish to participate may implement the APIs themselves. This means that ActionPack itself is agnostic as to the ORM used, and even supports hand-rolled models.

ActiveRecord

ActiveRecord has undergone a large internal change, integrating Nick Kallen’s ActiveRelation codebase. This replaces the internal ad-hoc query generation with query generation based on relational algebra. In essence, it means that you can chain queries for refinement before Rails builds the actual SQL query.

For instance, instead of Post.scoped immediately connecting to the database and retrieving all posts, it creates a relation object representing the idea of “all posts”. You can then refine further, such as Post.scoped.where(:author => "Yehuda"). This means that you can pass around relations, refining them as appropriate in helper methods before the view finally kicks off the query when it iterates over the objects.

This also unifies similar concepts in ActiveRecord, like named scopes, into a single relational concept that pervades the entire ORM.

ActiveSupport

In order to facilitate use of ActiveSupport by other libraries, ActiveSupport 3.0 makes it easy for developers to cherry-pick specific parts of the library they wish to use. In particular, we have made all dependencies between ActiveSupport files explicit, so that if you require a particular extension, you can be sure that you get everything that you need. In the past, in order to use ActiveSupport extensions reliably, you needed to require the full library.

ActiveSupport also comes with some new and retooled modules, such as ActiveSupport::Concern (which makes it easier to manage dependencies between modules) and a significantly faster ActiveSupport::Callbacks.

Railties

We made the biggest user-facing changes in Railties. A Rails 3 application is an object (of the class Rails::Application), which also doubles as a valid Rack application. Instead of configuring your application in a block, you configure the Application itself (which now lives in config/application.rb).

The Rails::Application class also inherits from an Engine class, making Engines behave almost identically to Applications, as appropriate. And Engines are a special kind of Railtie, again simplifying and unifying the concepts in the Rails glue code.

The rails command itself now has special powers inside of Rails applications, replacing the commands in script with subcommands of rails. For instance, use rails server instead of script/server.

In Conclusion…

It’s been an incredibly busy and productive year. Rails 3 is the culmination of thousands of man-hours by numerous dedicated open source contributors. It’s been a long journey, filled with debate, feedback, and writing code, refactoring it and then refactoring it yet again. While this release is just a beta, it represents the bulk of things to come in a stable and tested state.

Plugin authors should take a look at the new exposed APIs and start porting plugins to work on Rails 3. The community is excited, and we expect adoption to be rapid, aggressive and just as importantly, vocal. We’ll be listening to that feedback to help get Rails 3 to its final perfected state, and you should be listening too, so your plugins can make the jump with us.

Leave comments here and on the official Ruby on Rails blog; we’ll be here to respond!