CurbFu – Now With Killer Testing!

Matt Wilson and I have been making some awesome new updates to CurbFu – our convenient wrapper for Ruby’s curb gem – itself an wrapper around libcurl.

Our latest improvement revolves around integration testing. At Greenview Data, we have several Rack apps that talk to each other, including a Rails app. While all of code for the individual apps was unit tested, we still lacked a good integration test to make sure everything played well together.

In the past I had tried to set up a VM instance that was controlled by a series of RSpec Story Runner tests. Getting the test server instances updated and running was a pain, managing all of the running processes was a pain (especially testing a system with down processes), and writing tests to verify data became quite contrived.

With the introduction of Bryan Helmkamp’s Rack::Test and Rails 2.3’s support for Rack, I dreamed of having these apps running in memory, able to send HTTP requests to each other without having to set up a separate integration testing system. Now that all of our apps use CurbFu to send HTTP requests to each other, we seized the opportunity to create a testing module that override’s CurbFu’s get/post/put/delete operations. Instead of sending a real HTTP request over the internet, CurbFu will now call any rack apps you specify directly using Rack::Test.

Essentially, this system is similar to FakeWeb and Webrat’s newly added Rack support, but the difference is that if you use CurbFu for all of your HTTP communication, you can quickly and easily set up an integration test environment or set up smart mocks of HTTP services.

How does it work?

require ‘curb-fu’

CurbFu.stubs = {

‘a.example.com’ => RackApp1.new,

‘b.example.com’ => RackApp2.new

}

class RackApp1

def call(env)

CurbFu.get(‘http://b.example.com’)

end

end

class RackApp2

def call(env)

puts “WOW!”

end

end

CurbFu.get(‘a.example.com’)

#=> WOW!

require 'curb-fu'

CurbFu.stubs = {
  'a.example.com' => RackApp1.new,
  'b.example.com' => RackApp2.new
}

class RackApp1
  def call(env)
    response = Rack::Response.new
    response.status, response.body = [200, '']
    CurbFu.get('http://b.example.com')
    response.finish
  end
end

class RackApp2
  def call(env)
    response = Rack::Response.new
    response.status, response.body = [200, "WOW!"]
    response.finish
  end
end

puts CurbFu.get('a.example.com').body
#=> WOW!

Loading a Rails app is a bit more involved:

require 'curb-fu'
require '/path/to/rails/app/config/environment'
require 'dispatch'

CurbFu.stubs = {
  'my.cool.railsa.pp' => ActionController::Dispatcher.new
}

We’ve set up a series of Cucumber stories to test the interplay between our various rack apps. By having the rails app in memory, we’re able to access ActiveRecord objects from our step definitions, as if we’re within the rails app. We’re also able to stub specific methods on objects within our rack apps and we throw up quick little Rack apps that can pretend to be things like a Solr server or a site hosting an RSS feed. The sky is the limit!

Once caveat we’ve run into is the issue of namespacing. It may be a good idea to namespace your Rails model classes as they may conflict with other loaded Rack apps.

Have fun testing!

Links:
CurbFu

Double Negatives

Recently I was working on a project adding new functionality that would queue a long-running task to a background job. The original code looked something like this:

receive_data
forward_data

My task was to run enqueue_data instead of forward_data since the forwarding was taking too long. I wanted to retain the ability to run forward_data if a specific parameter was passed in. Since I was creating a parameter that would override a default feature, my first intuition was to name this parameter “noenqueue”:

receive_data
if params['noenqueue']
  forward_data
else
  enqueue_data
end

I then smacked myself in the forehead when I realized I had “pulled a Microsoft.” I had created a setting that is named such that setting it to true would disable a feature, while setting it to false would enable the feature. You see this all the time in Microsoft products.

It irritates me because you have to think in double-negative terms when setting the parameter. Instead, it is much more intuitive to create settings that are positive – that setting them to true will enable something.

Lesson learned: I renamed the parameter to “forward_immediately”:

receive_data
if params['forward_immediately']
  forward_data
else
  enqueue_data
end

One good syntactic feature of Ruby is the unless keyword. I prefer to use “unless variable.nil?” instead of “if variable” if I want to only operate on non-nil objects. I find it a bit more intuitive.

unless document.nil?
  document.write
end

However, you can still get into trouble when you need to branch:

unless document.nil?
  document.write
else
  document.new(str).write
end

An else branch on an unless is, in my opinion, the least readable or intuitive way to write branches.

Great Lakes Ruby Bash 2008

I’m really excited about the upcoming Great Lakes Ruby Bash in Ann Arbor on October 11th.  The distinguished Rubyists of the Grand Rapids Ruby Users Group are joining forces with us (Ann Arbor Ruby group) to put the conference together.  After attending eRubyCon in Columbus last weekend, I have to say the local conferences are much more engaging than the mega-conferences like RailsConf.  It’s great being able to get a chance to talk to everyone.  Hope to see you there!

BackgrounDRb 1.0.4 – Considerations

Today I gleefully updated BackgrounDRb to 1.0.4 after I learned that a new release was out that supported clustered BDRb servers.  I quickly learned that regiester_status has been removed.  That’s right.  All that code that sends data from your workers to your app need to be refactored to put data into cache[job_key].  This approach is more thread-safe, however.  Perhaps you could even write your own wrapper that puts data into cache on a call to register_status.

All in all, however, there are some really cool new features related to clustering.  I was starting to think for a while that BDRb had been abandoned and that Bj was the new way to go.  This release clearly puts BackgrounDRb on top for spawning asynchronous tasks.

Continuations in Ruby

A week ago, Dr. Frens spoke about continuations in Ruby at the Michigan Ruby Users Group. It was very interesting as it brought back my college days when I learned Scheme and functional programming concepts. I always enjoyed functional programming because it presents such a different way of thinking through problems.

I was going to blog about his presentation, but it turns out he blogged all his own notes already. But what I found interesting is the sort of hybrid nature that Ruby has. It is both an object-oriented language and it also supports some functional programming concepts. The drawback Dr. Frens found was that Ruby doesn’t do tail recursion optimization, so it’s not really worth using continuation passing style functions. Interesting, nonetheless!