Instrumenting excon

Note: Our very own Michael Brodhead recently wrote a blog post about excon and fog. With his permission, we’re reposting it here. At Engine Yard we make heavy use of fog to interface with back-end cloud providers. Fog in turn uses excon to make HTTP requests. A few features make excon a better choice than Net::HTTP:

• Better at reusing connections.
• Automatic retries for idempotent requests.
• Easy to stub for your unit tests.

My coworkers and I at Engine Yard needed to measure the performance of fog’s HTTP calls so we added one more feature to excon: instrumentation.

Enter ActiveSupport::Notifications

Larry Diehl called my attention to the new-fangled notification API in Rails 3, namely, ActiveSupport::Notifications. Rather than reinvent the wheel, said Larry, it made sense to use an existing API which at least some people would already be familiar with. ActiveSupport::Notifications is an implementation of our old friend, the publish-subscribe pattern and turns out to be pretty easy to use.

I won’t re-create the RDoc here, but the basic idea is you can instrument blocks of code by wrapping them in an #instrument call like so:

ActiveSupport::Notifications.instrument("cheezburger", :extra => :information) do
  # Do stuff.
end

Elsewhere in your app you can record those events by subscribing to them:

ActiveSupport::Notifications.subscribe(/cheezburger/) do |*args|
   puts "K THX BAI"
end

Now every time the first block is called, we’ll be notified with a call to the second block. Excellent. Now what?

With excon

When you include an instrumentor class in the Excon constructor you can then subscribe to excon’s events:

connection = Excon.new('http://icanhascheezburger.com/',
    :instrumentor => ActiveSupport::Notifications,
    :instrumentor_name => 'myapp.excon')

Elsewhere…

ActiveSupport::Notifications.subscribe(/myapp.excon/) do |*args|
  puts "Excon did stuff!"
end

If :instrumentor_name is omitted, the base name defaults to “excon”. Excon generates three different events: excon.request, excon.retry, and excon.error. Requests and retries each have associated durations, errors do not. By digging into args we can determine how long the request took along with other assorted goodness.

Too many gems!

Suppose you don’t want to include activesupport in your application. No problem. Simply define a class which responds to the #instrument method and record incoming events however you please.

class SimpleInstrumentor
  class << self
    attr_accessor :events

    def instrument(name, params = {}, &block)
      puts "#{name} just happened."
      yield if block_given?
    end
  end
end

connection = Excon.new('http://icanhascheezburger.com/', :instrumentor => SimpleInstrumentor)

Go forth and conquer

Hopefully I’ve given you enough information to start measuring excon for your own purposes. In a future post I’ll discuss how we used this excon feature to make pretty graphs of our fog calls to AWS.