JRuby 1.6 Released…Now What?

Hello again friends!

We’ve just wrapped up the longest, largest JRuby dev cycle ever: JRuby 1.6. It’s time to recap what we’ve accomplished and talk about what comes next…both for you and for JRuby.

###A Mountain of Work

Boy, and I thought JRuby 1.5 would never be topped. That release cycle last 5 months from the end of JRuby 1.4 maintenance in late 2009 until the 1.5 final release in May of 2010. We fixed an resolved an incredible 456 bugs and had around 1250 commits, making it our largest release at the time.

For JRuby 1.6, we “only” resolved 420 issues. But during the roughly 9-month dev time, we logged nearly 2500 commits (twice as many as JRuby 1.5). What the heck did we accomplish?

###New and Notable

I’m going to direct you to my JRuby 1.6.0 RC1 post to get more details, but here’s the “short” list of major release bullets. I will call out the fact that since the 1.6.0.RC1 announcement and today, we fixed an additional 150+ issues, most of them reported against our release candidates. Thank you for helping us make this an outstanding release!

• Ruby 1.9.2 API and language features: JRuby 1.6 is the first release where we recommend people start trying 1.9 mode. We’ve put in months of work making sure the new APIs function, we’ve updated the standard library to the current MRI 1.9.2 copy, and we’re actively looking for missing or buggy features to add or fix. If you’ve been waiting for 1.9 support in JRuby, now’s your time to give it a try…pass –1.9 at the command line or put it in the JRUBY_OPTS environment variable and you’re ready to go!
• Improved Ruby performance: As with every release, there’s tons of incremental perf improvements in JRuby 1.6. We never get as much time to focus on performance as we’d like, but this time there’s been more attention paid to execution performance specifically. Small benchmarks are running 1.5 to 2x faster, and CPU-bound applications should perform better as a result. We’ve also laid groundwork to enable our own optimizing compiler modes and to make it easier for the JVM to optimize things for us. It’s even better now, with enormous potential for JRuby 1.7.
• RubyGems updates and enhancements: We’ve bumped RubyGems up to version 1.5.1, which greatly improves startup time. We’ve disabled (by popular demand) the generation of ri and rdoc during gem installation. We’ve added support for installing Java libraries directly from Maven as if they’re regular gems. And you get all this in both 1.8 and 1.9 modes.
• Profiling and debugging: JRuby 1.6 improves the flat profiler from 1.5.6 (–profile or –profile.flat) and adds a graph-based profiler for more detail (–profile.graph). We’ve also resolved release issues for the ruby-debug gems, so you can gem install them from JRuby without hassle.
• Platform and native: This release marks the first time we’ve called Windows an “officially supported” platform. All that really means is we’ve got a Windows CI server and we’re keeping it green for all the suites we run on Linux and OS X. We’re also previewing experimental support for installing C extensions written to MRI’s C API. Not every extension works, and there’s certainly some caveats…but it’s a hell of a thrill seeing sqlite3_ruby or rubysdl installing and working.
• JVM and JVM languages: For 1.6, I wired up a lot more Ruby logic to the new “invokedynamic” JVM bytecode, which may enable us to run Ruby code many times faster than we do today. We also included enhancements to make it easier to call from Ruby into Scala libraries, which has enabled the Lift web framework to offer support for Ruby. The JVM is is becoming an amazing polyglot environment, and JRuby’s leading the charge.
• All-around improvements: JRuby starts up faster, runs faster, uses less memory, and integrates with the JVM and JVM libraries better. It behaves consistently across platforms, runs anywhere the JVM runs (including on Android!), and lets you remain a Rubyist while you take advantage of an amazing VM and solid libraries. It’s the best of both the Ruby and the JVM worlds, together in one package.

###What’s Next?

I won’t go all cliché on you and say this is just the beginning for JRuby. We’re widely accepted as an outstanding Ruby implementation (and an outstanding JVM language), and JRuby runs applications for banks, governments, airports, space exploration, and more. JRuby is no newcomer…so we’re well past the “beginning” stage.

What I will say is JRuby is about to cross a threshold. The release of JRuby 1.6 marks the first time we’re largely caught up with the released version of Matz’s Ruby, 1.9.2. Now of course development continues unabated on Ruby 1.9.3, and we’ll continue to chase MRI forever. But we’ve made up a lot of ground in a very short time…which means we’re going to have more freedom to work on things other than Ruby compatibility.

Now it gets exciting.

###Looking Toward the Future

We love working on JRuby because we love working atop the JVM. Say what you will about Java the language…the JVM is very hard to beat. There are JVMs for just about every platform out there. The JVM boasts some of the best garbage collection and optimizing JIT capabilities in the world. The platform plays host to tens of thousands of libraries, increasing the chance that you’ll find what you need…and that it will actually do what it claims. And best of all, when we work on JRuby we don’t have to worry about the JVM at all…there are dozens or hundreds of folks around the world ensuring every JVM is moving forward at a breakneck pace.

So is the JVM perfect? Of course not!

JRuby stands out among JVM languages for having done the most to accommodate the JVM. We’ve hacked around aging, cumbersome APIs (process management, IO, JDBC). We’ve gone “to the metal”, using native libraries and OS features to improve your JVM experience (posix bindings, FFI, C exts). I personally have gone to great lengths to make Ruby code not just compile to JVM bytecode, but compile in such a way that the JVM can optimize it well. So it may come as a surprise to hear how excited we are that Java 7 is going to make a lot of that work unnecessary and irrelevant.

###JRuby 1.7 and Java 7

I have a vision for JRuby 1.7. With our release cycles these days, we’ll probably see a JRuby 1.7 release some time around the release of Java 7. And I want JRuby to take as much advantage as possible of the features Java 7 brings to the table. What are these features, you ask?

###InvokeDynamic

Probably the biggest and most exciting change in Java 7 is the new support for dynamic invocation. Now I’m not just talking about being able to dispatch from A to B dynamically, like we have in Ruby. Obviously we and other languages have been doing that for years. What I’m talking about is finally being able to tell the JVM what the hell we mean by dynamic invocation…and teaching the JVM how to optimize it.

InvokeDynamic has been underway for three to four years now. It brings to the JVM two key features:

• The new “invokedynamic” bytecode, for making calls that are wired up at runtime rather than at compile time.
• The MethodHandle API, which provides optimized “function pointers” to wire up those runtime calls.

Optimizing VMs like the JVM do their best work when calls are as direct as possible. The JVM takes a call from A to B, sees that you’re always calling B, and optimizes both methods as if they were one. This is called “inlining”, and it can be done ahead-of time by compilers like GCC or at runtime by optimizing runtimes like the JVM. All the most important optimizations grow out of inlining. Conversely, if you can’t inline B into A, you’re not going to get the fastest-possible execution.

On Java 6 and earlier, JRuby has a hard time making dynamic calls inline. Because there’s a lot of method lookup logic between A and B, we confuse the JVM; it can’t see that A’s dynamic call eventually gets to B, and the two are never apart, so it treats them separately. We still get a great boost from the JVM’s optimizer, since A and B both make a lot of calls into the JRuby runtime. But we’re defeating most optimizations because we don’t have a good way to explain to the JVM “hey…ignore all the logic between A and B, and treat them as a direct call.”

That’s where invokedynamic comes in. With invokedynamic, we’re explicitly saying “this call is special”. The JVM sees the invokedynamic call and asks JRuby “ok…what do I do?” JRuby responds by wiring up a series of MethodHandle objects that take A’s call and wire it all the way to B. But instead of that intermediate logic confusing the JVM, it now knows all that logic is just in service of A’s dynamic call. It knows that B is being called from A, and the two should be optimized together. That’s really amazing.

I’ve seen this in action many times now. I’ve seen the JVM inline Ruby code into Ruby code, without any crazy compiler tricks in JRuby. Even more amazing, I’ve seen Ruby inline into Java and Java inline into Ruby. Yes, that’s right; with invokedynamic, you’ll see optimization happen across all JVM languages; it’s very likely some application out there will see Ruby, Clojure, Scala, and Java all inlined and optimized together. And there’s another incredible benefit for JRuby users: thousands of lines code and hundreds of thousands of bytes of bytecode specifically crafted to support dynamic calls will simply disappear…since invokedynamic eliminates the need for many tricks we utilized in the past.

There’s a very real chance that invokedynamic could improve JRuby performance many times, putting us on par with our statically-typed brothers like Java and Scala. And that means you can write Ruby code without fear. Awesome.

###New IO 2: Eclectic Bugaboo

The addition of InvokeDynamic also fits another theme: things that should have been in the JVM in the first place. NIO2 takes that theme and runs with it.

The “New IO” APIs, added in Java 1.4, fixed a long standing problem for Java developers: matching native IO performance and features. NIO added the ability to read from IO “Channels” directly into native buffers, rather than lifting bytes up to JVM objects. It made real nonblocking IO calls possible by abstracting the standard nonblocking APIs across all platforms (think eselect, kqueue, etc). And it finally gave us direct control over sockets, file streams, memory-mapped regions of disk, raw datagram traffic, and more. It made the JVM a solid platform for high-performance IO.

That was nine years ago. And little has changed since then.

In addition to wanting better control over IO, JVM users have screamed for better access to the filesystem itself. We’ve wanted to query and set permissions. We’ve wanted to monitor for filesystem events, like modified files and directories, without constantly polling. We wanted to be able to manipulate symlinks – seriously…SYMLINKS…totally unsupported in the JVM (except by intrepid upstarts like JRuby that have routed around the JVM.) NIO2 finally does that.

So, what do we get with NIO2?

• Platform-agnostic filesystem events: You read that right…JRuby users will be able to use filesystem event APIs that look and behave the same across all major platforms. No more futzing around with C extensions, no more platform-specific logic. Write once, run anywhere.
• Filesystem attributes: NIO2 will enable us to manage a much broader range of filesystem attributes across platforms. We’ll have all the permissions for UNIX plus APIs for managing Windows ACLs. We’ll be able to manage symbolic and hard links directly. And we’ll be able to read directories just like POSIX users have done for years.
• Filesystem inspection: There are several new APIs for walking the filesystem quickly, where previously a lot of cycles were wasted constructing Java File objects, examining them, and then proceeding. There’s APIs for quickly reading in file contents, reading in all lines from a file, or getting a direct channel to the file’s bytes on disk. And there’s an entire set of filesystem-general APIs for querying available space, attributes (remote, read-only, etc), and more.

###Speedier Adoption?

Java 7 also marks the first time that a major Java release will be available in open source from day 1.

OpenJDK is the GPLed version of Sun’s “Hotspot” JVM and class libraries. When the Sun piñata broke open, OpenJDK is one of the jewels that fell out. Because of OpenJDK, we now have Oracle, IBM, RedHat, Apple, Azul, and others all collaborating on the same JVM, ensuring it will run great on all platforms. We have top-notch JVMs for every Linux and BSD flavor. And I believe OpenJDK will mean Java 7 adoption moves much more quickly than Java 5 or 6.

All the best development these days are done by folks like you and me that aren’t afraid of new things. Give us a buildable, open-source project, and we’ll start exploring how we can use it. Because JVM releases used to lag behind on many platforms, and because it wasn’t possible to know…really know…what changed in the underlying codebase, people were usually terrified to make a move. Now that OpenJDK is out there, you can build the JVM on your own, you can understand how the platform works. And you can be confident in making a move to Java 7, taking advantage of everything it offers.

For JRuby users, this means you can start playing with solid OpenJDK7 preview releases right now, with a final release coming out this fall. You can start to take advantage of invokedynamic and NIO2. The platform is finally moving again…and it’s going to be a hell of a ride.

###A Bright Future

JRuby’s future is looking, well, amazing. The Java patform has emerged from a long slumber, and JRuby users will soon have an amazing array of features at their fingertips. You can still be a true Rubyist, writing Ruby code, and stand shoulder to shoulder with any language or runtime on performance and features. Most importantly: you can still have fun, and get work done at the same time!

It’s a great time to try JRuby.