Archive for the 'Coding' Category

CurbFu – Now With Killer Testing!

May 20, 2009

Matt Wilson and I have been making some awesome new updates to CurbFu – our convenient wrapper for Ruby’s curb gem – itself an wrapper around libcurl.

Our latest improvement revolves around integration testing. At Greenview Data, we have several Rack apps that talk to each other, including a Rails app. While all of code for the individual apps was unit tested, we still lacked a good integration test to make sure everything played well together.

In the past I had tried to set up a VM instance that was controlled by a series of RSpec Story Runner tests. Getting the test server instances updated and running was a pain, managing all of the running processes was a pain (especially testing a system with down processes), and writing tests to verify data became quite contrived.

With the introduction of Bryan Helmkamp’s Rack::Test and Rails 2.3’s support for Rack, I dreamed of having these apps running in memory, able to send HTTP requests to each other without having to set up a separate integration testing system. Now that all of our apps use CurbFu to send HTTP requests to each other, we seized the opportunity to create a testing module that override’s CurbFu’s get/post/put/delete operations. Instead of sending a real HTTP request over the internet, CurbFu will now call any rack apps you specify directly using Rack::Test.

Essentially, this system is similar to FakeWeb and Webrat’s newly added Rack support, but the difference is that if you use CurbFu for all of your HTTP communication, you can quickly and easily set up an integration test environment or set up smart mocks of HTTP services.

How does it work?

require ‘curb-fu’
CurbFu.stubs = {
‘a.example.com’ => RackApp1.new,
‘b.example.com’ => RackApp2.new
}
class RackApp1
def call(env)
CurbFu.get(’http://b.example.com’)
end
end
class RackApp2
def call(env)
puts “WOW!”
end
end
CurbFu.get(’a.example.com’)
#=> WOW!

require 'curb-fu'

CurbFu.stubs = {
  'a.example.com' => RackApp1.new,
  'b.example.com' => RackApp2.new
}

class RackApp1
  def call(env)
    response = Rack::Response.new
    response.status, response.body = [200, '']
    CurbFu.get('http://b.example.com')
    response.finish
  end
end

class RackApp2
  def call(env)
    response = Rack::Response.new
    response.status, response.body = [200, "WOW!"]
    response.finish
  end
end

puts CurbFu.get('a.example.com').body
#=> WOW!

Loading a Rails app is a bit more involved:

require 'curb-fu'
require '/path/to/rails/app/config/environment'
require 'dispatch'

CurbFu.stubs = {
  'my.cool.railsa.pp' => ActionController::Dispatcher.new
}

We’ve set up a series of Cucumber stories to test the interplay between our various rack apps. By having the rails app in memory, we’re able to access ActiveRecord objects from our step definitions, as if we’re within the rails app. We’re also able to stub specific methods on objects within our rack apps and we throw up quick little Rack apps that can pretend to be things like a Solr server or a site hosting an RSS feed. The sky is the limit!

Once caveat we’ve run into is the issue of namespacing. It may be a good idea to namespace your Rails model classes as they may conflict with other loaded Rack apps.

Have fun testing!

Links:
CurbFu

Double Negatives

January 27, 2009

Recently I was working on a project adding new functionality that would queue a long-running task to a background job. The original code looked something like this:

receive_data
forward_data

My task was to run enqueue_data instead of forward_data since the forwarding was taking too long. I wanted to retain the ability to run forward_data if a specific parameter was passed in. Since I was creating a parameter that would override a default feature, my first intuition was to name this parameter “noenqueue”:

receive_data
if params['noenqueue']
  forward_data
else
  enqueue_data
end

I then smacked myself in the forehead when I realized I had “pulled a Microsoft.” I had created a setting that is named such that setting it to true would disable a feature, while setting it to false would enable the feature. You see this all the time in Microsoft products.

It irritates me because you have to think in double-negative terms when setting the parameter. Instead, it is much more intuitive to create settings that are positive – that setting them to true will enable something.

Lesson learned: I renamed the parameter to “forward_immediately”:

receive_data
if params['forward_immediately']
  forward_data
else
  enqueue_data
end

One good syntactic feature of Ruby is the unless keyword. I prefer to use “unless variable.nil?” instead of “if variable” if I want to only operate on non-nil objects. I find it a bit more intuitive.

unless document.nil?
  document.write
end

However, you can still get into trouble when you need to branch:

unless document.nil?
  document.write
else
  document.new(str).write
end

An else branch on an unless is, in my opinion, the least readable or intuitive way to write branches.

How I Got Started Programming

July 13, 2008

Continuing the meme:

How old were you when you started programming?

Officially, it was my freshman year of high school.  Unofficially, the most “programming” I did was modifying config files to get my PC games to run faster on our Gateway 386.  A favorite game of mine growing up was “Rocky’s Boots” on our Tandy 286 which involved completing electronic circuit puzzles.  

 

 

How did you get started in programming?

I liked computers and my high school just happened to offer a BASIC programming class on old (even then) Apple //e’s.  After that I was hooked.  I think I inherited my dad’s creativity – his outlet is woodworking and construction.  I love to build things and doing it on a computer was the most logical step, since that’s where I spent all my time anyway.

My first programming job was a part-time gig during high school at an ad agency creating Access database applications for telemarketers’ data entry.

What was your first language?

Good old BASIC.  I quickly learned the evils of spaghetti code.  My final project for the BASIC class in high school was to do a “hi res” (80×48) animation.  It featured a bunny jumping through a static forest scene, a gun popping out from the side of the screen, shooting the bunny in the head.  Perhaps I was playing a bit too much Wolfenstein 3D?

Later on in high school, I moved through Pascal, C, and C++.  A past-time of mine in the computer lab was to create Myst-style dungeon crawlers using Hypercard.  There was always a secret spot to click to defeat the boss.  

What was the first real program you wrote?

How do you define “real?”  My first for-money program was a simple Access database for telemarketers.  Of course, there is a minimal amount of programming involved there.  My college classes offered me plenty of interesting projects.  One was a series of programs written to compute large matrix multiplications over distributed systems using various threading/multiprocessing libraries.  A part-time job I had during college consisted of small web apps (mainly shopping carts, CRM) written in ColdFusion.  

What languages have you used since you started programming?

BASIC, Pascal, C, C++, Scheme, Java, JavaScript, ObjectiveC, AppleScript, ColdFusion, T-SQL, PHP, C#, Visual Basic, Perl, and last, but not least, Ruby!

By far, Ruby and Scheme have been my favorites.  Ruby’s flexibility, friendliness, and the way it so nicely complements TDD/BDD have won me over.  While I’ve never “monetized” my knowledge of Scheme, it was so much fun to think “functionally” to solve a problem.  

What was your first professional programming gig?

Working for a contractor in Northern Virginia.  I worked on some C/C++ interfaces for USPS mail sorting machines and a system for nursing assistants to document nursing home care that involved a ColdFusion web app and a VoIP interface to an interactive voice system.  I learned quite a bit about the “real world” of programming.  At that time I was into creating “sugar castles” of code in my head and spent a lot of time fixing mistakes I thought I was incapable of making.  It was a good reality check.

If there is one thing you learned along the way that you would tell new developers, what would it be?

A multipart answer:

  1. Don’t typecast yourself into knowing one specific language or tool.  Chances are it will become obsolete and you will be “that guy” who maintains that dinosaur system and is simply there to punch the time clock and get a paycheck.  The world of computer science and software development is so much bigger and more interesting than just Java or Ruby or MVC or TDD.
  2. Be open.  Admit when you’re wrong.  Learn and improve.
  3. Get things done.  Wasting your time over-engineering a system will be simply that – a waste of time.  Your customers don’t care how cool your algorithms or languages are and you’re better off spending that time testing and refining the user experience – delivering what the customer actually wants.
What’s the most fun you’ve ever had… programming?
My languages course in college – writing Scheme code.  Again, the functional paradigm is a lot of fun.  Hacking with others on fun side projects over pizza and beer is always a blast.  It’s the geek’s version of building a shed in your backyard with your buddies or jamming together in your basement.

Continuations in Ruby

December 12, 2007

A week ago, Dr. Frens spoke about continuations in Ruby at the Michigan Ruby Users Group. It was very interesting as it brought back my college days when I learned Scheme and functional programming concepts. I always enjoyed functional programming because it presents such a different way of thinking through problems.

I was going to blog about his presentation, but it turns out he blogged all his own notes already. But what I found interesting is the sort of hybrid nature that Ruby has. It is both an object-oriented language and it also supports some functional programming concepts. The drawback Dr. Frens found was that Ruby doesn’t do tail recursion optimization, so it’s not really worth using continuation passing style functions. Interesting, nonetheless!

Eager Loading

December 1, 2007

One thing that made me uneasy about Rails’ ActiveRecord was how it handles associations. For instance, say I have a table of products in the database and another table listing comments about the products. So, for each product we have many comments.

Back in the good ol’ ColdFusion days, I would write a query that joins the two tables. Then I would loop through each field and have some funky conditional statements, like the following:

<cfset prevProductId = 0 />
<cfloop query="qryProductsWithComments">
 <cfif product_id neq prevProductId>
  ...if this is not the first product, finish printing the previous product's template...
  ...we've hit either the first product or a new product, so print out the beginning of the product template and the first comment...
 <cfelse>
  ...this is another comment, so print out just another comment...
 </cfif>
 <cfset prevProductId = product_id />
</cfloop>

That’s pretty darn confusing, but it was better, performance-wise, than executing a new query every time you iterate over the products loop, like so:

<cfloop query="qryProducts">
 ...spit out the product template...
 <cfquery name="qryComments" ...>
  SELECT * FROM comments WHERE product_id = #qryProducts.product_id#
 </cfquery>
 <cfloop query="qryComments">
  ...spit out a comment...
 </cfloop>
</cfloop>

By default, ActiveRecord will perform similarly to the latter example. However, with eager loading, you can get the same performance as the former example without all of the confusing if statements:


# in our model:
class Product < ActiveRecord::Base
 has_many :comments
end


# in our controller:
class ProductsController < ApplicationController
 def index
  @products = Product.find(:all, :include => :comments)
 end
end


# in our view:
<% for product in @products -%>
 <%= render :partial => "products/product" %>
 <% for comment in @products.comments -%>
  <%= render :partial => "comments/comment" %>
 <% end -%>
<% end -%>

So much cleaner! And easier!

With ASP.NET 2.0, you’re pretty much stuck doing the equivalent of the second CF example using the server controls or you have to build your own object data source that will work similarly to ActiveRecord, but you’ve just spent an hour or so reinventing the wheel.

Some Very Strange Markup

November 7, 2007

I was viewing the source of the Facebook iPhone webapp to see how they did their fancy transitions between pages, when I found some very strange HTML:

<body home=”home.php”>
<div class=”toolbar1 tops chrome”>
<u href=”home.php”><img src=”images/facebook.png”/></u>
<u class=”button plain leftButton” href=”#status”>Status</u>
<u class=”button plain” href=”#search” radio=”true”>Search</u>
</div>

How can a <body> tag have a “home” attribute or a <u> tag have an “href” attribute? I scoured the W3C spec and looked for a possible mobile HTML standard supporting it, but couldn’t find anything.Well, it turns out they’re doing some non-standard trickery by adding custom attributes to their elements. They’re then using JavaScript to get the values of those attributes and use them in their “onclick” handlers for each element. Sneaky!

Fun With Microformats

October 24, 2007

I just updated my food blog by encoding all of the restaurants’ contact information in microformats. It’s basically a way to encode vCard/vCal data using HTML. It’s part of the latest movement to bring semantics (meaningful data that can be interpreted by computer programs) to web content. For instance, with this technology you could easily write a program to find all the phone numbers and addresses listed on the internet and generate a worldwide phone book. Or you could track who has lived in a given house over the past X years. Or you could generate statistics on how many CEOs live next to golf courses. Or you could figure out, according to published calendars, when most people will be away from home and thus not watching the latest sitcom. And that’s just the beginning.

One thing that’s cool about the hCard specification is its flexibility. According to the spec, they recommend the following format:

<div class="vcard">
  <span class="fn n">
     <a class="url" href="http://t37.net">
       <span class="given-name">Fréderic</span>
       <span class="family-name">de Villamil</span>
     </a>
  </span>
  <span class="nickname">neuro</span>
  <a class="email" href="mailto:neuroNOSPAM@t37.net">
     <span class="type">pref</span><span>erred email</span>
  </a>
  <span class="org">Omatis</span>
  <span class="adr">
     <abbr class="type" title="dom">France</abbr>
     <span class="type">home</span> address
     <abbr class="type" title="postal">mail</abbr> and
     <abbr class="type" title="parcel">shipments</abbr>:
     <span class="street-address">12 rue Danton</span>
     <span class="locality">Le Kremlin-Bicetre</span>
     <span class="postal-code">94270</span>
     <span class="country-name">France</span>
  </span>
</div>

But I was able to easily use different elements in place of <div> and <span>:

<p class="vcard">
  <span class="fn n">
     <a class="url" href="http://t37.net">
       <span class="given-name">Fréderic</span>
       <span class="family-name">de Villamil</span>
     </a>
  </span>
  <span class="nickname">neuro</span>
  <a class="email" href="mailto:neuroNOSPAM@t37.net">
     <span class="type">pref</span><span>erred email</span>
  </a>
  <span class="org">Omatis</span>
  <a class="adr" href="http://somegooglemapslink">
     <abbr class="type" title="dom">France</abbr>
     <span class="type">home</span> address
     <abbr class="type" title="postal">mail</abbr> and
     <abbr class="type" title="parcel">shipments</abbr>:
     <span class="street-address">12 rue Danton</span>
     <span class="locality">Le Kremlin-Bicetre</span>
     <span class="postal-code">94270</span>
     <span class="country-name">France</span>
  </a>
</p>

This was useful because wordpress does not allow <div> in your blog posts. It automatically converts them to <p> :p