CoffeeScript is happening…

…whether you like it or not. Personally, I’m psyched — it looks like it solves a lot of the annoyances I’ve had with Javascript syntax, and I got a sweet deal on CoffeeScript: Accelerated JavaScript Development during the conference, which I hope to put to good use starting this week.

The Rails Community is Awesome

My wife’s response after she read my tweets upon my return:

“Just wondered, who is this guy? Dinner anyone? Share a cab? You became an extrovert on your mothership.”

It’s true, I did. It’s weird. I’ve never been to Baltimore before (save time spent between flights at BWI), but among my fellow Rails coders, I’ve never felt more “at home.”

As I thought about all of the people I finally got to thank in person for all they’ve done to make my time in the Rails community more enjoyable, it really drove home what an awesome community we have. It’s easy to lose sight of this while working away in Louisville, my little corner of Kentucky.

Aside from this, getting a chance to put a real face to a name, make and receive thank-yous for all the hard work we put in on Rails patches, plugins, and the like, has the effect of re-energizing a developer. I’d go on and list all of the awesome people I got to meet in person here, but I think it’d just make @casron more jealous.

I would like to say, however, that Emilio Tagua (@miloops) has got to be one of the friendliest guys I’ve ever met. I’d hoped to catch up with him to thank him for helping me get commits added to ARel back in the 1.x days that were needed for MetaSearch and MetaWhere. Instead, he ended up two seats over from me at Ignite, noticed I was hacking on a Squeel bug, and introduced himself first. Every time I saw him during the remainder of the conference, he had a smile and a kind word. Just an all-around great guy to have met.

PostgreSQL is awesome, too!

Now, I should have known this. I had started out using MySQL many years ago, when it had solid speed advantages over PostgreSQL, but far fewer features. Somewhere along the line, that balance shifted, but I stuck with MySQL due to familiarity. Nick Gauthier’s KnowSQL presentation has made a believer out of me. I’ll be using PostgreSQL as my RDBMS of choice from here on, and making sure that Squeel works as well as possible with it, too.

The Fundamentals are ALWAYS Relevant

There were tons of talks on design patterns, best practices, and so on, and even as long as I’ve been doing this whole coding thing, there are plenty of areas in which I can improve. In particular, Avdi Grimm’s talk, Confident Code, José Valim’s talk about the design principles behind the Rails 3 refactoring, and the talk on Building Bulletproof Views by John Athayde and Bruce Williams all challenged me to elevate my game in these areas.

HTML5 is sooooo much more than semantic tags

LocalStorage, audio, video, canvas drawing, web sockets, web workers — this stuff is all crazy cool and I can’t wait until more browsers support it. Mike Subelsky’s HTML5 tutorial was a real eye-opener.

That’s about it, for now. What were some of your takeaways from RailsConf this year?

The 15th annual Webby Award winners were announced yesterday. Some say The Webby Award is the Internet’s most respected symbol of success. We’re just delighted to receive Honoree status for our work and rank in the top ten percent of the nearly 10,000 entrants. We’re in good company too, Disney and Lego also received the Honoree nod.

Website and mobile winners were selected for recognition based on excellence in Content, Structure and Navigation, Visual Design, Functionality, Interactivity and Overall Experience. Winners were selected by The International Academy of Digital Arts and Sciences (IADAS), a global collection of experts and technology innovaters, including David Bowie, Arianna Huffington, Harvey Weinstein, Martha Stewart, Vinton Cerf, Biz Stone.

We’re proud to be recognized by our peers with this gesture but we’re even more proud of our work and the success of Figment. Tell us what you think…

If your best argument against using Ruby is a perception that it’s slow, a reminder: Moore’s Law is on our side. :)

Ruby’s been fighting the “Ruby is slow” meme for years now, and not without good reason — every day, new strides are being made in one Ruby implementation or another that show there is ample opportunity for efficiency improvements.

That being said, do the people making this argument realize how old and tired it is? I don’t mean, “bashing Ruby for being slow is so 2002.” I mean, “bashing programming languages for being slow is so 1972 (at least).”

“Ruby is slow. Java is better.”
“Java is slow. C is better.”
“Forget about compilers, they’re inefficient. Program in assembly!”

Resisting a programming language because it’s perceived to be “slow” is one of the most short-sighted decisions one could ever make. Over time, VMs get more efficient, compilers produce better machine language, and bandwidth, storage, and compute cycles become cheaper and more abundant. However, I have yet to hear of a single developer who has succeeded in performing more than one hour of work per hour.

Choose the programming language that is “fast enough” for your needs, maximizes your source code’s expressiveness, and increased your developers’ happiness and productivity, because:

1. “Fast enough” isn’t nearly as fast as you think it is. The dissenting developer in the story above, who rejected Ruby because it was slower than Java, is writing a web-based document management system that needs to support user concurrency in the tens, not thousands.
2. A project with a highly expressive codebase will be easier to develop and maintain.
3. It’s hard enough to attract and retain strong developers even when you aren’t hamstringing their productivity.

And one last reminder: It’s easy to write non-performant code in any language. Sometimes it pays to check up on the carpenter before you blame the hammer.

Treetop appeared to gather all the concepts together into an understandable domain specific language. All of the tokenization and node structure can go into a single file, and the interactive nature of Ruby makes for the perfect sand box. I felt like I could get somewhere if I invested just an hour into this. I was happy to find that my impressions were correct.

After a short time I had caught on enough to start writing my own code. Once over the hump the rest was easy. I was able to write and test a Treetop grammar for parsing CSV files within a few hours. I chose CSV parsing because I was already familiar with the format, and I could compare my implementation to not just one but two existing Ruby libraries.

commaseparatedfile.treetop

grammar CommaSeparatedFile
  rule lines
    line (newline line)*
    {
      def values
        vals = []
        vals << line.values
        more.elements.each { |additional| vals << additional.line.values }
        vals
      end
    }
  end

  rule line
    valueline / emptyline
  end

  rule emptyline
    '' {
      def values
        []
      end
    }
  end

  rule valueline
    leading:(value separator)* trailing:value
    {
      def values
        list = []
        list += leading.elements.collect { |lead| lead.value.text_value } if leading
        list << trailing.text_value if trailing
        list
      end
    }
  end

  rule value
    quotedvalue / nakedvalue
  end

  rule quotedvalue
    '"' wrapped:( !'"' . / '""' )* '"' {
      def text_value
        wrapped.text_value.gsub(/""/,'"')
      end
    }
  end

  rule nakedvalue
    (!(separator / newline ) .)*
  end

  rule separator
    ','
  end

  rule newline
    [\n]
  end
end

One concept that became clear to me is that the tokens that are defined in the grammar are objects in the resulting tree returned after input is parsed. Ruby code embedded in the Treetop’s node definitions are added as methods to the object declared by that node.

In the simple harness below, the lines object returned from the parse call below is the object representation of the rule lines node above, defined as the root of the parse tree. The values method I call below is defined above in the definition of the lines node.

require 'rubygems'
require 'treetop'
require 'commaseparatedfile' 

parser = CommaSeparatedFileParser.new
lines = parser.parse("this,that,these,those")
puts lines.values.inspect #-> [["this", "that", "these", "those"]]

I think it was much simpler for me to write this parsing as a grammar instead of trying to combine regular expressions with Ruby code. The grammar above handles multiple lines and quote-wrapped values, and escaped quotes.

It was actually fun to write as well, and I’m still learning a lot. Notice that my lines node and my valueline node both approach a similar problem (a list of 0 or more items) differently. For the lines the grammar grabs one line followed by 0 or more separators followed by lines. For the valueline I grab 0 ore more values followed by separators and terminate on the last value. Is one approach better than the other? Are they both junk? I can’t wait to figure that out.

My poor parser is left in the dust by both Ruby’s stdlib csv library, and the fastercsv gem. I ran these tests under the 1.8.5 version of Ruby. The dataset I used is 44k in total and has 961 records.

                       user     system      total        real
stdlib_csv:        0.210000   0.000000   0.210000 (  0.217275)
faster_csv:        0.040000   0.000000   0.040000 (  0.060788)
treetop_grammar:   2.910000   0.010000   2.920000 (  3.126020)

I don’t know what sort of speed I should reasonably expect, actually. I’d hoped that my compiler would be competitive with the standard library’s csv parsing. It’s possible that my grammar’s definition could be innefficient. If anyone has any ideas let me know.

After playing around with Treetop I’ll definitely tinker some more. There are many other features I haven’t explored yet, like extending a base grammar to create a new grammar. PEGs have implementations in other languages so this isn’t a Ruby-only facility, either.

On a side note, I used stdlib’s benchmark to generate the numbers above. Pretty swank stuff.

require 'rubygems'
require 'treetop'
require 'commaseparatedfile'
require 'fastercsv'
require 'csv'
require 'benchmark'

def with_stdlib_csv(csv)
  parsed = CSV.parse(csv)
  parsed.size
end

def with_fastercsv(csv)
  rows = FasterCSV.parse(csv)
  rows.size
end

def with_treetop_parser(csv)
  parser = CommaSeparatedFileParser.new
  parsed = parser.parse(csv)
  parsed.values.size
end

csv = File.read('../kentucky.csv')

Benchmark.bmbm("treetop_parser:".length) do |x|
  x.report("stdlib_csv:") { with_stdlib_csv(csv) }
  x.report("faster_csv:") { with_fastercsv(csv) }
  x.report("treetop_grammar:") { with_treetop_parser(csv) }
end

cap --set-before branch=testbranch  deploy

…where ‘testbranch’ is the name of my subversion branch, and ‘deploy’ is the action to take. The ‘–set-before option’ sets a variable before the recipes are loaded (sort of). All you need to do to make this work you just need to set the repository URL correctly based on if the branch variable exists or not. In my case, that is editing the top of deploy.rb in the config section of my rails app:

if variables.include?(:branch)
  set :repository,  "http://svn.example.com/project/branches/#{branch}"
else
  set :repository,  "http://svn.example.com/project/trunk"
end

Here is a simple example:

We’ll start with a table named libris_book that contains book entries. The name libris_book won’t really fly with the Rails pluralized table name approach. It’s easy, however, to fix that disconnect in the model:

class Book < ActiveRecord::Base
  set_table_name 'libris_book'
end

Let’s take a look at our data:

~/src/libris$ script/console
Loading development environment.
>> Book.count
=> 1230

Good enough. Now, we’ll be good little coders and write some unit tests using fixtures. We’ll create a fixture file and our test class:

books.yml

test_book:
  title: Life in Lawrenceburg
  author: Darren Day

book_test.rb

require File.dirname(__FILE__) + '/../test_helper'

class BookTest < Test::Unit::TestCase
  fixtures :books

  def test_fixtures
    assert_equal(2,Book.count)
    test_book = books(:test_book)
  end
end

Looks good, but it won’t work!

~/src/libris$ rake test
(in /home/darrend/src/libris)
/usr/local/bin/ruby -Ilib:test "/usr/local/lib/ruby/gems/1.8/gems/rake-0.7.2/lib/rake/rake_test_loader.rb" "test/unit/book_test.rb"
Loaded suite /usr/local/lib/ruby/gems/1.8/gems/rake-0.7.2/lib/rake/rake_test_loader
Started
E
Finished in 0.01133 seconds.

  1) Error:
test_fixtures(BookTest):
ActiveRecord::StatementInvalid: Mysql::Error: Table 'libris_test.books' doesn't exist: DELETE FROM books
...

Ugh. That doesn’t look pretty. After some googling around the following comes together

• Rename books.yml to libris_book.yml. The fixture yaml file has to share the same name as the database table name.
• Change the fixtures declaration to fixtures :libris_book.
• Use set_fixture_class to connect the fixture to the model class.

Here is our new and improved book_test.rb file:

require File.dirname(__FILE__) + '/../test_helper'

class BookTest < Test::Unit::TestCase
  set_fixture_class :libris_book => Book
  fixtures :libris_book

  def test_fixtures
    assert_equal(2,Book.count)
    test_book = libris_book(:test_book)
  end
end

At first I decided my test would just respond with the yamlized request (just like the mongrel sample did). This worked fine but there was a significant size difference in the two documents (WEBrick’s was much bigger). In the end, I just decided to respond with a few bytes of text.

The Code

mongrel_service.rb:

require 'rubygems'
require 'mongrel'
require 'yaml'

class SimpleHandler < Mongrel::HttpHandler
  def process(request, response)
    response.start do |head,out|
      head["Content-Type"] = "text/html"
      out << "I am mongrel, hear me roar"
    end
  end
end

config = Mongrel::Configurator.new :host => "localhost", :port => 4000 do
  listener {uri "/", :handler => SimpleHandler.new}
  trap("INT") { stop }
  run
end

config.join

webrick_service.rb:

require 'webrick'
require 'yaml'
include WEBrick

class SimpleServlet < HTTPServlet::AbstractServlet
  def do_GET(request, response)
    response["Content-Type"] = "text/html"
    response.body = "I am WEBrick, hear me roar"
  end
end

s = HTTPServer.new( :Port => 4001 )
s.mount("/", SimpleServlet)
trap('INT'){ s.shutdown }
s.start

The Test

There are serveral good benchmarking tools available, but I had apache bench (ab) handy, so that is what I used. I just used the stock gem install of mongrel and the WEBrick that comes with ruby 1.8.4 (the one non-stock change I made was to comment out access logging for WEBrick since mongrel doesn’t log by default).

The Results

The Conclusion

Looks like mongrel it is….

So not only can you lookup the current volume of Google stock in a spreadsheet (=GoogleFinance(“GOOG”, “volume”)) or find out Roger Clemens’s ERA (=GoogleLookup(“Roger Clemens”,”earned run average”)) you can access those values real time using an open API. I cobbled together a quick ruby program that can retrieve values from a private or public (published) Google spreadsheet.

The public sample should work for everyone since I published that spreadsheet for the world to see. You will need to provide your own credentials and spreadsheet key to use the authenticated example. I will caution that this is just a proof of concept that has limited functionality, weak parsing, and no error checking.

#!/usr/bin/env ruby

require 'net/http'
require 'net/https'
require 'uri'
require 'rubygems'
require 'hpricot'

#
# Make it east to use some of the convenience methods using https
#
module Net
  class HTTPS < HTTP
    def initialize(address, port = nil)
      super(address, port)
      self.use_ssl = true
    end
  end
end

class GoogleSpreadSheet
  def initialize(spreadsheet_key)
    @spreadsheet_key = spreadsheet_key
    @headers = nil
  end

  def authenticate(email, password)
    url = URI.parse('https://www.google.com/accounts/ClientLogin')
    response = Net::HTTPS.post_form(url,
      {'Email'=>email,
       'Passwd'=>password,
       'source'=>"ruby-ss-example-1",
       'service'=>'wise' })
    @headers = {
     'Authorization' => "GoogleLogin auth=#{response.body.split(/=/).last}",
     'Content-Type'  => 'application/atom+xml'
    }
  end

  def evaluate_cell(cell)
    path = "/feeds/cells/#{@spreadsheet_key}/1/#{@headers ? "private" : "public"}/basic/#{cell}"
    doc = Hpricot(request(path))
    result = (doc/"content[@type='text']").inner_html
  end

  private
  def request(path)
    http = Net::HTTP.new('spreadsheets.google.com', 80)
    response,data = http.get(path, @headers)
    data
  end
end

gs = GoogleSpreadSheet.new('p39irgfu5fsFokTPyPCG8Lg')
puts gs.evaluate_cell("A1")
puts gs.evaluate_cell("A2")
puts gs.evaluate_cell("B1")

gs = GoogleSpreadSheet.new(<your spreadsheet key>)
gs.authenticate(<your email>, <your password>)
puts gs.evaluate_cell("E132")

The fun really just starts here. You can edit existing spreadsheets too (although you can ‘t create or delete them yet). As Rich pointed out to me a few days ago, it won’t be too long before we are adding ‘export to Google spreadsheet’ functions to our web applications.

rake tmp:sessions:clear # Clears all files in tmp/sessions