Building Structured API Clients with API Smith

Darcy Laycock is a web developer from Perth, Western Australia specialising in Ruby on Rails development. By day Darcy attends the University of Western Australia and works for The Frontier Group, a web development company. By night Darcy is hard at work building cool stuff. He’s also a humble 2011 Ruby Hero.

Whilst prototyping the early stages of a new app with Filter Squad, we found ourselves prototyping a lot of API clients for new versions of APIs that were lacking up to date clients or in the case of other APIs, were missing functionality we required.

As our starting tool set, we started with a fairly standard / typical API toolset - HTTParty for the HTTP portion and Hashie for the basic rapid prototyping data structures. As we prototyped more and more APIs, we started to see a clear pattern in how these two portions were interfacing and we started to see ourselves reinventing the same portions.

To this end, Steve Webb and I wrote a library called ‘API Smith’ - a simple tool set combining HTTParty and Hashie in a manner that we found was incredibly fast and powerful to prototype with. By combining several simple concepts together, API Smith aims to fulfill three basic requirements:

1. Converting API responses into Ruby objects should be a straight forward and simple process.
2. Extracting parts of a response shouldn’t be complex.
3. HTTP is your friend and should be simple.

Based on these three things, today we’re going to briefly show how to build part of an API client for Clicky, a fairly robust analytics suite. For this cut down example, we’re going to model the tallies methods as noted in the Clicky API docs.

Before we do anything, you’ll want to first get api_smith. This is a simple matter of running:

gem install api_smith

Step 1. Declaring your data types

First, to get a better feel for the data we’re working with, we want to model the output response. From the Clicky docs, under the “Responses” section we see how the Clicky API will return the data to us In the API Smith side portion, we’re lucky because API Smith’s mapping objects - APISmith::Smash (for “Smarter Hash”) are fairly easy to map to the incoming data. In this example, we can simply subclass the API Smith class and then use the property class method to declare what our possible fields are:

require 'rubygems'
require 'api_smith'

SMART_NUMBER_TRANSFORMER = lambda { |v| v =~ /^\d+$/ ? Integer(v) : v }

class TallyStat < APISmith::Smash
  property :title
  property :value,         :transformer => SMART_NUMBER_TRANSFORMER
  property :value_percent, :transformer => :to_f
  property :url
  property :clicky_url
  # Goal Information
  property :incompleted, :transformer => :to_i
  property :conversion,  :transformer => :to_f
  property :revenue
  property :cost
end

If you’ve worked with Hashie::Dash before, this will look reasonably familiar. The primary change that API Smith introduces that underpins most of the application design is that of ‘transformers’ - Namely, in this context a transformer is simply an object that responds to the call method and returns ‘transformed’ data.

In the example code above, you can see that we provide the :transformer option on the property method. In the :value field, we pass it a lambda (the basic form of a call-able object) and in the next three we pass it a symbol representing a method name. Under the hood, API Smith will convert the symbol in to a proc and will apply it only when a value is given for said property.

To test it out, fire up IRB, paste in the code and then run:

p TallyStat.new(:value => '1000', :clicky_url => 'http://example.com')

And you should see the output of our object. Likewise, as a short cut out APISmith::Smash subclasses also define call as a class method, making it possible to pass a subclass (e.g. Our TallyStat class) to another object as a smart transformer, e.g.:

property :tallies, :transformer => TallyStat

Along with this, APISmith::Smash is also smart enough to convert arrays and the like when called as a transformer.

Step 2. Writing the HTTP Client

The next step is to write a HTTP client by using the APISmith::Client mixin. Much like the normal HTTParty mixin, this gives us handy methods on the class like get and post, but in the case of API Smith we’re also given the option to configure endpoints (separate from the base_uri), a fairly simple hierarchical way of declaring query and body parameters as well as specifying request options for HTTParty.

Secondly, it also adds tools to unpack and transform data. Similar to the :transformer method on the property class method for APISmith::Smash subclasses, it also has a :transform option (With similar rules - Essentially, anything that has a #call method will used to take the raw response and convert it into a usable object.

In addition to this, it provides a :response_container method that accepts an array of keys and will extract the data from the response before passing it to the transformer.

To get started, add the following below your code from the original code:

class ClickyClient include APISmith::Client

class TallyStatCollection < APISmith::Smash property :type property :date property :dates, :transformer => lambda { |c| c.map { |v| TallyStat.call(v[‘items’]) }.flatten } end

TALLY_METHODS = %w(visitors visitors-unique actions actions-average time-average time-average-pretty bounce-rate visitors-online feedburner-statistics)

class Error < StandardError; end

base_uri ‘http://api.getclicky.com/’ endpoint ‘api/stats/4’

attr_reader :site_id, :site_key

def initialize(site_id, site_key) @site_key = site_key @site_id = site_id add_query_options! :site_id => site_id, :sitekey => site_key end

TALLY_METHODS.each do |m| define_method m.tr(‘-‘, ‘_’) do |*args| api_tally_call m, *args end end

  private

  def check_response_errors(response)
    if response.first.is_a?(Hash) and (error = response.first['error'])
      raise Error.new(error)
    end
  end

  def base_query_options
    {:output => 'json'}
  end

  def api_tally_call(type, options = {})
    get '/', :extra_query => {:type => type}.merge(options),
      :response_container => [0], :transform => TallyStatCollection
  end

end

This will declare a client that not only accepts a site id and key as parameters (e.g. client = ClickyClient.new(1234, 'your-site-key')) but that will also:

• Correctly unpack errors (e.g. if an invalid api response is returned, it will raise a ClickyClent::Error exception).
• It will pass through any parameters in the query string
• It will automatically return a collection of tally stats, grouped by a date.

Whilst it’s a pretty basic example, if you try it out with some real credentials (hint: see the clicky api docks for an example site id and site key), you’ll see it works as expected:

c = ClickyClient.new('siteid', 'sitekey')
p c.visitors

Which should print out something similar to:

<#ClickyClient::TallyStatCollection dates=[<#TallyStat value=129>] type="visitors">

Conclusion

As you’ve seen in this rapid fire tutorial, we’ve managed to built a fairly simple client for a portion of the Clicky API. More importantly, even though it’s still reasonably brittle and missing support for many options in the Clicky API, in a relatively short time we were able to not only pull back data from API calls but transform them into structured objects we can further manipulate.

If you wish to take it further, there are a few more things you can implement to extend it into a more complete Clicky API client:

• Support for converting option types (e.g. date ranges)
• Better support for extracting data (e.g. getting daily data over a period of X days versus just the current day)
• Support for more API methods
• Actual proper tests

I’ve put a very rough version I wrote on GitHub here so you can take a look, but I suggest having a play personally - maybe there is a small app you use that has an API for which you could build a client using API Smith.