Tuesday, February 1, 2011

Iteration 2, Tues

A long morning retrospective is an unintuitive event to schedule, but sometimes a good idea, and my pair and I still got a nice chunk of work done before lunch. As part of the learning process, our team has been leveling up to the point where we self-start our internal processes without trainer (i.e. boss) supervision.

The ruby gem roo imports data from spreadsheets into wherever you want to send it- usually a database of your own. Some experimentation with it today revealed two important things:

  1. roo 1.9.3 does not appear to install its own dependencies. When using roo, write a script to take care of that.
    Dependencies of roo:
    bones (3.6.2)
    builder (3.0.0)
    google-spreadsheet-ruby (0.1.2)
    little-plugger (1.1.2)
    loquacious (1.7.0)
    nokogiri (1.4.4)
    oauth (0.4.4)
    rake (0.8.7)
    ruby-ole (1.2.11.1)
    rubyzip (0.9.4)
    spreadsheet (0.6.5.1)
  2. roo can import data from .xls or .xlsx and parse it into the variable types that we need.
An alternative to roo is Apache POI, which is written in Java and thus will interface with our system through fewer layers than roo would have to. A third alternative which we may or may not explore is to use the java bindings provided by OpenOffice. 

When it comes time to use these with parts of the real dataset, we may expedite redumping the .xls by dumping csv and checking md5sums to validate the cache. i.e, find /path/to/spreadsheets -name '*.xls' |md5sum > checksums.txt, and then use a diff to compare to the last run. 

To open any file in mac terminal, open filename.extension will accomplish the task. 

Quote: "learning what you're dong rather than how to do something" (in relation to how to find and adjust paths of inputs and outputs rather than how to do so for a specific program)

In connection with roo, I spent some time getting several haphazard installs of rvm into a more understandable state, setting a default ruby (1.8.6 because of the OSX support) and examining the different places where gems can be installed to, and the possibility of automagically-generated symlinks from all installed rubies to the system ruby's gem folder. However, it seems too complex for our current problem, and perhaps unwise in general if versions of gems become a relevant concern. 

IntelliJ library dependency filepath complexities are deceptively simple, and some of the error messages are verbose but unilluminating because in this case the desired classpath seems to match the existing classpath but the error persists, which leads me to believe that the error is symptomatic of a different problem. Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/xmlbeans/XmlException