Documenting one-off parsers

Posted about 7 years back at Saaien Tist

A lot of day-to-day work consists of parsing data files to transform the contents from one format into another or to create statistics. However, when you have to get back to those scripts at a later stage - you need something similar in another project or you notice that something along the way must have gone horribly wrong - it can often be quite hard to figure out what the script actually did. Having meaningless script filenames like ParseBlast.rb doesn't help either. (Parse BLAST into what?)

I must say that things improved a lot when I switched from perl to ruby. Trying to understand a perl script that I wrote a couple of weeks earlier was a real pain, while I normally have no problems understanding ruby code that I wrote months ago... Has a lot to do with the simple syntax and expressiveness (is that a word?) of that language.

But just understanding the code is not enough. To be able to assess if you could use a script from an earlier project in a new one, you often also have to get hold of information in that script that is not inherently encoded in its code: what was the project? What did the input look like? What did the output have to look like? That's where the script documentation comes in, because having that information easily available greatly reduces the time you need to assess if you should copy-paste-adapt the script, or just start from scratch.

What I try to do, is always use the same template when starting a new parsing script. Even when the parsing itself would only take 5 lines, I try to use this whole template. A quick walk-through:

Lines 2-32: The documentation, consisting of:

  • script name
  • short usage message
  • a description of what the script does
  • list of arguments that can be used
  • input format
  • output format
  • contact information
Lines 38-40: Definitions of classes that are used later on in the script itself
Lines 42-51: Parsing the arguments to the script. If the user (i.e. me when I run it) uses --help or an argument that does not exist, he automatically gets the documentation of the script. If I would use the -a tag here, the output to the screen would be

USAGE
-----
./this_script.rb [ -h | --help ]
[ -i | --infile | < ] your_input.txt [ -o | --outfile | > your_output.txt ]


Lines 53-58: Logging of the options that were used in running the script.
Lines 60-70: Create the input and output streams.
Lines 72-75: The actual parsing code. This is the bit that does the work (using the classes described in lines 38-40).
Lines 77-79: Clean up.

Here's the complete template.

#!/usr/bin/ruby
# == NAME
# this_script.rb
#
# == USAGE
# ./this_script.rb [ -h | --help ]
# [ -i | --infile | < ] your_input.txt # [ -o | --outfile | > your_output.txt ]
#
# == DESCRIPTION
# Description of what this script does...
#
# == OPTIONS
# -h,--help:: Show help
# -i,--infile=INFILE:: Name of input file. STDIN if not defined.
# -o,--outfile=OUTFILE:: Name of output file. STDOUT if not defined.
#
# == FORMAT INPUT
# >gi|4531835|bla
# ACTTACCGACCGACTGACTACTTATGCCA
# >gi|4861534|blabla
# CTACCCCATCTACCGGGGCTCGACT
# ...
#
# == FORMAT OUTPUT
# 4531835 29
# 4861534 25
# ...
#
# == AUTHOR
# my full contact information

require 'rdoc/usage'
require 'optparse'
require 'ostruct'
require 'logger'

### Define classes here
class MyClass
end

### Get the script arguments and open relevant files
options = OpenStruct.new()
opts = OptionParser.new()
opts.on("-h","--help",
"Display the usage information") {RDoc::usage}
opts.on("-i","--infile", "=INFILE",
"Input file name") {|argument| options.infile = argument}
opts.on("-o","--outfile", "=OUTFILE",
"Output file name") {|argument| options.outfile = argument}
opts.parse! rescue RDoc::usage('usage')

log = Logger.new(File.new('this_script.log', File::WRONLY | File::TRUNC | File::CREAT))
log_level = Logger::INFO # or: DEBUG, WARN, FATAL, UNKNOWN

log.info('Script this_script.rb started')
log.info('Options:')
log.info(options.to_yaml)

if options.infile
input_stream = File.open(options.infile)
else
input_stream = $stdin
end

if options.outfile
output_stream = File.new(options.outfile,'w')
else
output_stream = $stdout
end

### Actually do some stuff
input_stream.each_line do |line|
output_stream.puts line
end

### Wrap everything up
output_stream.close
input_stream.close

(If I wanted this a bit DRYer, I'd have a constant with the script name at the top so I wouldn't have to repeat that filename over and over. To be done...)

If I wanted to over-organize, I'd create a little database with the descriptions of those scripts so I can search through them. Even though that would have it's use from time to time, that would be taking it too far. Having that documentation and a standard way of providing the command line arguments is enough for me for now.

I must admit I let this slip in the last couple of months, which doesn't mean it didn't work the months before that. That's just what happens when you go on holiday and completely forgot the habit of doing this afterwards.

UPDATE: InfiniteRed blogged about a similar approach later.

Episode 59: Optimistic Locking

Posted about 7 years back at Railscasts

When two people attempt to update the same record near the same time, one of the updates will likely be overwritten. You can solve this problem with optimistic locking.

Low Pro Behaviours 101

Posted about 7 years back at danwebb.net - Home

Okay, I’ve gone on about this for long enough and made lots of promises. So now’s time to make a start on explaining all this business with Low Pro and ‘behaviours’.

Behaviours are part of Low Pro and are object orientated mechanism by which you can handle events and maintain the state of an element (or indeed a set of elements) and as I’ve found out recently are an incredibly nice way of writing and deploying UI widgets. Before I dive in to how to write your own behaviours, let’s get a flavour for how to use them. If you want to play along you’ll need the latest versions of Prototype and Low Pro as well as the remote behaviours which you can grab from my SVN.

The first thing to note about a behaviour is that they are simply specialised constructor functions so you can use them with the new keyword to create an instance:

var aLink = $('edit_5');
var ajaxLink = new Remote.Link(aLink, { method: 'post' });

Behaviours, when used with the new keyword, always take an element reference as the first argument. Any subsequent arguments are passed to the behaviours constructor. In the case above, Remote.Link is a behavior which when attached to a link will make the link trigger an Ajax request rather than a tradtional page refresh. With this behaviour the second argument is an options hash that is passed through to the Ajax request. Now we’ve created an instance of the behaviour that instance of the behaviour is bound to that element for it’s life time on the page handling click events on the link and sending Ajax requests.

If you need to attach a behaviour to multiple elements on then using new could get a little tedious. To this end behaviours work very nicely with Event.addBehavior:

Event.addBehavior({
  'a.edit': Remote.Link
});

Using a behaviour with Event.addBehavior in this way will attach a seperate instance of the behavior to every element that matches the CSS selector. Notice we don’t pass in an event in the selector as you might be used to. The behaviour knows what events it wants to listen for. Now all the links in the page with the class name ‘edit’ will invoke Ajax request. You may need to pass in extra information to the behaviour as we did above with the options hash. Thanks to a bit of metaprogramming hackery we can do this too:

Event.addBehavior({
  'a.edit': Remote.Link({ onComplete: triggerEdit })
});

As you can see, it’s almost like CSS for interactivity where you’re defining how elements behave in an declaritive, almost code-less fashion. In my own projects I’ve gained massive advances in maintainability by separating everything out into behaviours then using Event.addBehavior as the glue to my HTML. You don’t need to use custom HTML attributes or weird classnames – you just bind behaviours to elements as you would style rules in CSS.

Of course, using this method you don’t get references to the behaviour instances you’ve created. In fact, you’ll hardly ever need to get references to these behaviour instances as most behaviors don’t need public methods at all but in case you need to get at them there’s a property of each behavior that contains an array of all its instances on the page:

Remote.Link.instances //=> an array of all the instances of Remote.Link

So that’s basically how to use behaviours. In a day or two I’ll post part two of Behaviours 101 which will show you how to make your own but in the meantime try out some of the behaviours I’ve already created. So far there’s a Draggable, Droppable, A date picker, Remote link and Remote form. Check out the demo page.

Open Source Rails & Flex eCommerce Application

Posted about 7 years back at work.rowanhick.com

[Announcement] Rowan Hick Consulting, Canada and Ron Hanley of Fastmount LTD, New Zealand are pleased to annouce that their new Rails/Flex/MySQL eCommerce application will be released to the community under an Open Source license. The application, is a successor to a PHP/MySQL based application developed by Rowan Hick and James McGlinn of Nerdsinc Ltd New Zealand. After nearly 2 years in production, with the advent of Rails, Flex, and the rapidly growing business needs of Fastmount, the current application no longer meets the business needs of Fastmount. We made the decision early this year to re-write a new application and have been developing requirements for the application as well as evaluating technologies. The application is a business to business eCommerce system. Allowing a marine manufacturing company in New Zealand to service it's agents, distributors and customers world wide. The existing application facilitates order management from creation through to shipping tracking, along with customer management and some reporting elements. Why are we open sourcing ? We made the conscious decision that we don't want to sell the application, as such being closed and proprietary we gain no extra business value by keeping it internal (this doesn't preclude the possibility of a hosted version being provided down the road). The application contains no proprietary information to Fastmount Ltd so there is no harm by open sourcing it. Having done a number of eCommerce solutions, the problems faced and solutions tend to be very similar regardless of domain. Thus the app's tend to look very similar. Why reinvent the wheel, over and over again? My personal vision is that a vibrant community will flourish around the application. If it can become an option for people when putting together an eCommerce solution then I would be ecstatic. Currently I haven't seen an active Rails based open source eCommerce engine out there so there's hope that this may become one of many possible standard 'engines' for people to use. Through our background in the widget ordering domain - solving business problems beyond the simple shopping cart such as; multiple price catalogues, multiple currencies, product/ package combination, agents, sales by regions etc - this may help to be a learning tool for those who aren't familiar with the problems at hand, introducing real world solutions beyond your standard learning materials and tutorials. Also by opening it up, and allowing public scrutiny will likely ensure that a higher standard of coding is reached within the application - there's nothing like judgement from your peers to keep those nasty kludges out of the codebase. Why Flex ? Flex, does at first glance seem to be an odd coupling with Rails in terms of philosophical approach. However as a business tool, this application needs to work perfectly, regardless of browser incompatibilities. In my own opinion, the time saved by not having to deal with these issues, along the extra interface richness granted by Flex, warrants the extra upfront time in creating the Flex interface. What should be noted here is this application is more of a business to business, rather than business to consumer type system - thus we can expect less casual browsing/ordering as in the case of a product catalog/ecommerce app versus a very defined business process for ordering. Having said all this, whilst the app has a Flex front end by using some Rails niceties there will be no stopping anyone developing an HTML front end for the app (infact the app will already have some front end work in HTML), so the core Rails application will become the engine and any various front end(s) that may appear can be bolted on. We're using a REST based approach to facilitate this. Roadmap Current Status: Initial development of Order management stories; Creating, editing, listing orders and customers. Evaluating implementing microframeworks for the Flex front end (Cairngorm or Model Glue Flex). Evaluating communication methods (HTTPService, WebOrb, RubyAMF). Release 1: Production ready system - Revamp of existing system in Rails and Flex, all existing user stories maintained. Streamlined order entry system, customer management, order payment processing (via ActiveMerchant), shipment tracking (via UPS). Release 2: Customer Relationship Management enhancements Release 3: Inventory Management, manufacturing enhancements. Community Release The application will be released to the community under an Open Source license (license to be decided), the only caveat from our production system will be licensed assets (such as icons) stripped out for community release with placeholders for people to find their own. I haven't as yet investigated a trac (or trac like system) that's impervious to spam, so for any community members that have any ideas reading this post feel free to drop us a comment. This will likely happen in the next month. Community Contributions We will welcome contributions and commit back into the core, so long as they don't negatively impact core functionality. Process as yet undecided. To ensure fair attribution - the credits page of the app will include your name as a contributor. Whatever license is decided upon will require that any modifications are publicly available as open source. Share and share alike principles. The Name? Undecided - if you want to throw out an idea feel free! Keep Informed All announcements will be made here on this blog. Keep informed, keep your RSS readers locked to http://feeds.feedburner.com/rowanhick In closing I'm personally very excited about providing this to the community and hope good things will come out of it.

Continuing Facebook Applications with Ruby On Rails

Posted about 7 years back at Liverail - Home

This is the continuing tutorial from the last post Tutorial on developing a Facebook platform application with Ruby On Rails where we can take the social recipe application and add some more Facebook features including posting to the feed, profile boxes, profile actions, the Facebook Query Language and sending out invitations to a user’s friends to come join in the cooking fun (oh boy). So lets get down to it, I’m presuming you have been through the first tutorial and have a working Facebook application. Even if you havn’t got it deployed today’s examples will work with an IFRAME application also.

UPDATE: This article is a bit out of date now, I have now posted an article on Rails, resful_authentication and Facebook connect

This is the continuing tutorial from the last post Tutorial on developing a Facebook platform application with Ruby On Rails where we can take the social recipe application and add some more Facebook features including posting to the feed, profile boxes, profile actions, the Facebook Query Language and sending out invitations to a user’s friends to come join in the cooking fun (oh boy). So lets get down to it, I’m presuming you have been through the first tutorial and have a working Facebook application. Even if you havn’t got it deployed today’s examples will work with an IFRAME application also.

Before we get into it Evan Weaver has posted two really useful things for Facebook and Ruby on Rails. First a way to output the Facebook exceptions nicely when viewing in Facebook and Second how to develop an application locally while tunneling from a server Facebook can see.

Sending events to the feed.

One of the features that makes Facebook so great is the powerful News Feed’s and mini-feed which keeps you updated on what’s going on with your friend’s lifes and in-touch with your application content. The Facebook API allows you to do two things with the feed. You can publish a News Feed story to just the user logged in with

    
fbsession.feed_publishStoryToUser(:title => "", :body => "", :image_1 => "")
    

or publish a Mini-Feed story to the user and a News Feed story to their friends with

    
fbsession.feed_publishActionOfUser(:title => "", :body => "", :image_1 => "")
    

In both cases you can publish up to four images in the story with will be strung to a 75×75 square by Facebook and cached and served by them. Additionally the stories may not show up depending on the quality of the other competing stories in the feed! Applications are also limited to calling feed_publishStoryToUser once every 12 hours for each user and feed_publishActionOfUser to 10 times in 48 hours for each user.

Now to our application. We are going to notify the user and all their friends when they create a new recipe so all their friends can go check it out and cook it. So in recipes_controller and the create action.

    
if @recipe.save

  fbsession.feed_publishActionOfUser(:title => "<fb:name/> 
  has added a recipe for <a href=\"http://apps.facebook.com#{url_for(:controller => 'recipes', :action => 'show', :id => @recipe, :only_path => true)}\" >#{@recipe.title}</a>", 
   :body => "#{@recipe.summary}")
    

In this case we use a url_for to add the URL to an absolute path. This is necessary for Facebook as all links on the pages outside your application must be absolute. We also have used a little FBML to display the user’s name in the news feeds.

Now it would be great if we could use observations on ActiveRecord models to do this but the fbsession is available only in the controllers, a limit of rfacebook

Adding a profile box to a user’s profile page

The profile box is a little mini-view of your application which is on a user’s profile page The quirk behind profile boxes is the don’t send a request to your application to display the content. Instead you must set what content (in FBML) will appear in your application box for each user. This is a push method rather than a pull method. More information here.

So when do you actually set the profile box content. Well it depends what you are putting in the box, if you are just updating it with information about the users data then you may want to update that users profile box after a certain event. If you have information from a users friends data you may need to update a users profile box and update any of their friends after a certain event.

In our case the profile box is going to contain just a list of the recipes a user has added, but we are going to keep it keep generic. Try adding this to the ApplicationController

    
def update_profile(uid = fbsession.session_user_id)
  #find the user specific recipes
  recipes = Recipe.find_my_recipes(uid)

  #render a partial to a string
  profile_box = render_to_string(:partial => 'users/profile_fbml', :locals => { :uid => uid, :recipes => recipes })

  #send the profile FBML to Facebook through the API for this user
  fbsession.profile_setFBML({:markup => profile_box, :uid => uid})
end
    

We can then call update_profile from any controller to update a users profile, by default it will update the current user but we can override this to say update a friends profile.

For the partail we need to create app/views/users/_profile_fbml.rhtml

    
<fb:if-is-own-profile>
  <fb:subtitle>You have created <%=pluralize(recipes.size, 'recipe')%></fb:subtitle>
  <fb:else>
   <fb:subtitle>
    <fb:name uid="<%=uid%>"/> has created <%=pluralize(recipes.size, 'recipe')%>
   </fb:subtitle>
  </fb:else>
</fb:if-is-own-profile>
<%recipes.each do |recipe| %>
  <div>
    <h2><a href="http://apps.facebook.com<%= url_for :controller => 'recipes', :action => 'show', :id => recipe%>">
    <%=h(recipe.title)%></a></h2>
    <p><%=h recipe.summary %></p>
  </div>
<%end%>

<fb:if-is-app-user>
<fb:else><a href="http://www.facebook.com/add.php?api_key=28e9f67df073c92829aa003e6762e074">
    Add Social Recipe for yourself</a></fb:else>
</fb:if-is-app-user>
    

This FBML code works in two ways, first if the user is viewing there own profile it says You have created but if someone else if viewing their profile it says Stuart Eccles has created for instance. We then link to the recipes, note the use of absolute URLs, a must on the profile box. Finally if another user has not added the application we chuck in a sneaky plug to our application. Note the api_key you will want to move this to a constant.

Finally we need to call the update_user and we will do so after a create or delete action on a controller i.e.

    
def update
  @recipe = Recipe.find(params[:id])

  if @recipe.update_attributes(params[:recipe])
   flash[:notice] = 'Recipe was successfully updated.'

   update_profile
   redirect_to :action => 'show', :id => @recipe
  else
   render :action => "edit" 
  end
end
    

Adding a profile action to a user’s profile page

A profile action appears as a link just under a users picture on their profile. The profile action link should be set at the same time as the profile box so to add a link we just need to add a bit of FBML to the end of the _profile_fbml.rhtml

    
<fb:if-is-own-profile>
<fb:profile-action url="http://apps.facebook.com<%= url_for :controller => 'recipes', :action => 'my'%>">
View My Recipes (<%=recipes.size%>)
</fb:profile-action>
<fb:else>
<fb:profile-action url="http://apps.facebook.com<%= url_for :controller => 'recipes', :action => 'index'%>">
View <fb:pronoun uid="<%=uid%>" possessive="true" capitalize="true"/> Recipes (<%=recipes.size%>)
</fb:else>
</fb:profile-action>
</fb:if-is-own-profile>
    

Again it gets switched depending on if the user is viewing their profile or someone else. If it is someone else we use the fb:pronoun to output His or Her.

Adding an application invitation

Our application is so cool no your users are going to want to share it with their friends (well yours will be, this tutorial application lacks coolness). So lets allow users to invite their friends.

We only want people to be inviting their friends how havn’t already added the application, unfortunately we don’t have an API call for this. Fortunately there is a way around this but we have to use the Facebook Query Language which is just like a select SQL statement. We can select data from the user, friend, group, group_member, event, event_member, photo, album, photo_tag tables. We can then call the API through fql_query method on fbsession, this will return XML we can then parse. You can try out your FQL first at the test console.

We need to create an invites controller:

    
script/generate controller invites         
    
    
class InvitesController < ApplicationController

  def select
    fql =  "SELECT uid, name FROM user WHERE uid IN" +
    "(SELECT uid2 FROM friend WHERE uid1 = #{@current_fb_user_id}) " +
    "AND has_added_app = 0" 
    xml_friends = fbsession.fql_query :query => fql
    @friends = Hash.new
    xml_friends.search("//user").map do|usrNode| 
      @friends[(usrNode/"uid").inner_html] = (usrNode/"name").inner_html
    end
    render_facebook
  end

end
    

In this code we select friends of the current user where the friends has_added_app = 0 so therefore are not using the application. We then parse their names and uids into a hash. Lets create two outputs for this list, a normal HTML one and an FBML one.

app/views/invites/select.rhtml

    
<% form_tag :action => 'send_invites' do -%>
<ul>
<% @friends.each do |uid,name|%>
     <li><input type="checkbox" name="friends[]" value="<%=uid%>"/> <%=name%></li>
<%end%>
</ul>

You can only invite a maximum of ten people at once.
<input type="submit" value="Invite"/>
<% end -%>
    
app/views/invites/select_fbml.rhtml
    
<% form_tag :action => 'send_invites' do -%>
<ul>
<% @friends.each do |uid,name|%>
     <li><input type="checkbox" name="friends[]" value="<%=uid%>"/><fb:profile-pic uid="<%= uid %>" size="thumb" /> <%=name%></li>
<%end%>
</ul>

You can only invite a maximum of ten people at once.
<input type="submit" value="Invite"/>
<% end -%>
    

The only difference is in the FBML version we have outputted a picture. You can style this in table or any way you want really but Facebook will only let you send 10 notifications at once and only 20 a day for each user. Now we need the send_invites action added to the controller

    
      def send_invites
        invite = render_to_string(:partial => 'invites/invite_fbml', :locals => { :inviter => @current_fb_user_id })

        friends = params[:friends]
        if friends.is_a? String
          invitees = friends
        else
          invitees = friends.values.join(",")
        end

        result_xml = fbsession.notifications_sendRequest(:to_ids => invitees, :type => 'Social Recipe', :content => invite, :invite => true, :image => "http://apps.facebook.com/socialrecipe/P1010825_tiny.jpg")

        response = CGI.unescapeHTML((result_xml/"notifications_sendrequest_response").inner_html)

        if response =~ /^(http|https):\/\/[a-z0-9]+([\-\.]{1}[a-z0-9]+)*\.[a-z]{2,5}(:[0-9]{1,5})?(\/.*)?$/ix
          #need to do a confirmation redirect
          if in_facebook_canvas?
            render :text => "<fb:redirect url=\"#{response}\"/>", :layout => false
          else
            redirect_to response
          end
        else
          flash[:notice] = 'Invites sent.'
          redirect_to :controller => 'recipes', :action => 'index'
        end
      end
    

Similar to the profile box we are using a partial with render_to_string to create the FBML to send in the notification. We pass in the inviter uid to use as a local in the partial. Note that the image parameter is required in the API call even if you don’t have an image to show! Once we have a response from the API call we will either be returned a URL to redirect the user to confirm the notification or it will return nothing at all. The redirect has to be handled in different way depending on if you are in the canvas, if you are you need to output the fb:redirect tag.

If it returns nothing if either means no confirmation is necessary and the notification was sent or there was an error….. hmmmm lets be optimistic and say it was sent shall we.

    
You have been invited to upload your recipes to Social Recipe! 
<fb:name uid="<%= inviter%>" firstnameonly="false" shownetwork="false"/> wants you to add Social Recipes so you can share recipes together
<fb:req-choice url="http://www.facebook.com/add.php?api_key=916c58dd4fba835cc43389a432fa98d0" label="Add" />
<fb:req-choice url="http://apps.facebook.com/socialrecipe" label="Go to Application" />
    

The fb:req-choice will give the user in the notification the options to add the application or go to it. If you now submit the form

Wrapup

So that’s all, there are a few more things to discover in the application interface but you should have all the basic elements to create some fun apps to share with your friends and their friends. There is certainly not enough error handling in here for a production application but i’ll leave that up to the reader.

Continuing Facebook Applications with Ruby On Rails

Posted about 7 years back at Liverail - Home

This is the continuing tutorial from the last post Tutorial on developing a Facebook platform application with Ruby On Rails where we can take the social recipe application and add some more Facebook features including posting to the feed, profile boxes, profile actions, the Facebook Query Language and sending out invitations to a user’s friends to come join in the cooking fun (oh boy). So lets get down to it, I’m presuming you have been through the first tutorial and have a working Facebook application. Even if you havn’t got it deployed today’s examples will work with an IFRAME application also.

UPDATE: This article is a bit out of date now, I have now posted an article on Rails, resful_authentication and Facebook connect

This is the continuing tutorial from the last post Tutorial on developing a Facebook platform application with Ruby On Rails where we can take the social recipe application and add some more Facebook features including posting to the feed, profile boxes, profile actions, the Facebook Query Language and sending out invitations to a user’s friends to come join in the cooking fun (oh boy). So lets get down to it, I’m presuming you have been through the first tutorial and have a working Facebook application. Even if you havn’t got it deployed today’s examples will work with an IFRAME application also.

Before we get into it Evan Weaver has posted two really useful things for Facebook and Ruby on Rails. First a way to output the Facebook exceptions nicely when viewing in Facebook and Second how to develop an application locally while tunneling from a server Facebook can see.

Sending events to the feed.

One of the features that makes Facebook so great is the powerful News Feed’s and mini-feed which keeps you updated on what’s going on with your friend’s lifes and in-touch with your application content. The Facebook API allows you to do two things with the feed. You can publish a News Feed story to just the user logged in with

    
fbsession.feed_publishStoryToUser(:title => "", :body => "", :image_1 => "")
    

or publish a Mini-Feed story to the user and a News Feed story to their friends with

    
fbsession.feed_publishActionOfUser(:title => "", :body => "", :image_1 => "")
    

In both cases you can publish up to four images in the story with will be strung to a 75×75 square by Facebook and cached and served by them. Additionally the stories may not show up depending on the quality of the other competing stories in the feed! Applications are also limited to calling feed_publishStoryToUser once every 12 hours for each user and feed_publishActionOfUser to 10 times in 48 hours for each user.

Now to our application. We are going to notify the user and all their friends when they create a new recipe so all their friends can go check it out and cook it. So in recipes_controller and the create action.

    
if @recipe.save

  fbsession.feed_publishActionOfUser(:title => "<fb:name/> 
  has added a recipe for <a href=\"http://apps.facebook.com#{url_for(:controller => 'recipes', :action => 'show', :id => @recipe, :only_path => true)}\" >#{@recipe.title}</a>", 
   :body => "#{@recipe.summary}")
    

In this case we use a url_for to add the URL to an absolute path. This is necessary for Facebook as all links on the pages outside your application must be absolute. We also have used a little FBML to display the user’s name in the news feeds.

Now it would be great if we could use observations on ActiveRecord models to do this but the fbsession is available only in the controllers, a limit of rfacebook

Adding a profile box to a user’s profile page

The profile box is a little mini-view of your application which is on a user’s profile page The quirk behind profile boxes is the don’t send a request to your application to display the content. Instead you must set what content (in FBML) will appear in your application box for each user. This is a push method rather than a pull method. More information here.

So when do you actually set the profile box content. Well it depends what you are putting in the box, if you are just updating it with information about the users data then you may want to update that users profile box after a certain event. If you have information from a users friends data you may need to update a users profile box and update any of their friends after a certain event.

In our case the profile box is going to contain just a list of the recipes a user has added, but we are going to keep it keep generic. Try adding this to the ApplicationController

    
def update_profile(uid = fbsession.session_user_id)
  #find the user specific recipes
  recipes = Recipe.find_my_recipes(uid)

  #render a partial to a string
  profile_box = render_to_string(:partial => 'users/profile_fbml', :locals => { :uid => uid, :recipes => recipes })

  #send the profile FBML to Facebook through the API for this user
  fbsession.profile_setFBML({:markup => profile_box, :uid => uid})
end
    

We can then call update_profile from any controller to update a users profile, by default it will update the current user but we can override this to say update a friends profile.

For the partail we need to create app/views/users/_profile_fbml.rhtml

    
<fb:if-is-own-profile>
  <fb:subtitle>You have created <%=pluralize(recipes.size, 'recipe')%></fb:subtitle>
  <fb:else>
   <fb:subtitle>
    <fb:name uid="<%=uid%>"/> has created <%=pluralize(recipes.size, 'recipe')%>
   </fb:subtitle>
  </fb:else>
</fb:if-is-own-profile>
<%recipes.each do |recipe| %>
  <div>
    <h2><a href="http://apps.facebook.com<%= url_for :controller => 'recipes', :action => 'show', :id => recipe%>">
    <%=h(recipe.title)%></a></h2>
    <p><%=h recipe.summary %></p>
  </div>
<%end%>

<fb:if-is-app-user>
<fb:else><a href="http://www.facebook.com/add.php?api_key=28e9f67df073c92829aa003e6762e074">
    Add Social Recipe for yourself</a></fb:else>
</fb:if-is-app-user>
    

This FBML code works in two ways, first if the user is viewing there own profile it says You have created but if someone else if viewing their profile it says Stuart Eccles has created for instance. We then link to the recipes, note the use of absolute URLs, a must on the profile box. Finally if another user has not added the application we chuck in a sneaky plug to our application. Note the api_key you will want to move this to a constant.

Finally we need to call the update_user and we will do so after a create or delete action on a controller i.e.

    
def update
  @recipe = Recipe.find(params[:id])

  if @recipe.update_attributes(params[:recipe])
   flash[:notice] = 'Recipe was successfully updated.'

   update_profile
   redirect_to :action => 'show', :id => @recipe
  else
   render :action => "edit" 
  end
end
    

Adding a profile action to a user’s profile page

A profile action appears as a link just under a users picture on their profile. The profile action link should be set at the same time as the profile box so to add a link we just need to add a bit of FBML to the end of the _profile_fbml.rhtml

    
<fb:if-is-own-profile>
<fb:profile-action url="http://apps.facebook.com<%= url_for :controller => 'recipes', :action => 'my'%>">
View My Recipes (<%=recipes.size%>)
</fb:profile-action>
<fb:else>
<fb:profile-action url="http://apps.facebook.com<%= url_for :controller => 'recipes', :action => 'index'%>">
View <fb:pronoun uid="<%=uid%>" possessive="true" capitalize="true"/> Recipes (<%=recipes.size%>)
</fb:else>
</fb:profile-action>
</fb:if-is-own-profile>
    

Again it gets switched depending on if the user is viewing their profile or someone else. If it is someone else we use the fb:pronoun to output His or Her.

Adding an application invitation

Our application is so cool no your users are going to want to share it with their friends (well yours will be, this tutorial application lacks coolness). So lets allow users to invite their friends.

We only want people to be inviting their friends how havn’t already added the application, unfortunately we don’t have an API call for this. Fortunately there is a way around this but we have to use the Facebook Query Language which is just like a select SQL statement. We can select data from the user, friend, group, group_member, event, event_member, photo, album, photo_tag tables. We can then call the API through fql_query method on fbsession, this will return XML we can then parse. You can try out your FQL first at the test console.

We need to create an invites controller:

    
script/generate controller invites         
    
    
class InvitesController < ApplicationController

  def select
    fql =  "SELECT uid, name FROM user WHERE uid IN" +
    "(SELECT uid2 FROM friend WHERE uid1 = #{@current_fb_user_id}) " +
    "AND has_added_app = 0" 
    xml_friends = fbsession.fql_query :query => fql
    @friends = Hash.new
    xml_friends.search("//user").map do|usrNode| 
      @friends[(usrNode/"uid").inner_html] = (usrNode/"name").inner_html
    end
    render_facebook
  end

end
    

In this code we select friends of the current user where the friends has_added_app = 0 so therefore are not using the application. We then parse their names and uids into a hash. Lets create two outputs for this list, a normal HTML one and an FBML one.

app/views/invites/select.rhtml

    
<% form_tag :action => 'send_invites' do -%>
<ul>
<% @friends.each do |uid,name|%>
     <li><input type="checkbox" name="friends[]" value="<%=uid%>"/> <%=name%></li>
<%end%>
</ul>

You can only invite a maximum of ten people at once.
<input type="submit" value="Invite"/>
<% end -%>
    
app/views/invites/select_fbml.rhtml
    
<% form_tag :action => 'send_invites' do -%>
<ul>
<% @friends.each do |uid,name|%>
     <li><input type="checkbox" name="friends[]" value="<%=uid%>"/><fb:profile-pic uid="<%= uid %>" size="thumb" /> <%=name%></li>
<%end%>
</ul>

You can only invite a maximum of ten people at once.
<input type="submit" value="Invite"/>
<% end -%>
    

The only difference is in the FBML version we have outputted a picture. You can style this in table or any way you want really but Facebook will only let you send 10 notifications at once and only 20 a day for each user. Now we need the send_invites action added to the controller

    
      def send_invites
        invite = render_to_string(:partial => 'invites/invite_fbml', :locals => { :inviter => @current_fb_user_id })

        friends = params[:friends]
        if friends.is_a? String
          invitees = friends
        else
          invitees = friends.values.join(",")
        end

        result_xml = fbsession.notifications_sendRequest(:to_ids => invitees, :type => 'Social Recipe', :content => invite, :invite => true, :image => "http://apps.facebook.com/socialrecipe/P1010825_tiny.jpg")

        response = CGI.unescapeHTML((result_xml/"notifications_sendrequest_response").inner_html)

        if response =~ /^(http|https):\/\/[a-z0-9]+([\-\.]{1}[a-z0-9]+)*\.[a-z]{2,5}(:[0-9]{1,5})?(\/.*)?$/ix
          #need to do a confirmation redirect
          if in_facebook_canvas?
            render :text => "<fb:redirect url=\"#{response}\"/>", :layout => false
          else
            redirect_to response
          end
        else
          flash[:notice] = 'Invites sent.'
          redirect_to :controller => 'recipes', :action => 'index'
        end
      end
    

Similar to the profile box we are using a partial with render_to_string to create the FBML to send in the notification. We pass in the inviter uid to use as a local in the partial. Note that the image parameter is required in the API call even if you don’t have an image to show! Once we have a response from the API call we will either be returned a URL to redirect the user to confirm the notification or it will return nothing at all. The redirect has to be handled in different way depending on if you are in the canvas, if you are you need to output the fb:redirect tag.

If it returns nothing if either means no confirmation is necessary and the notification was sent or there was an error….. hmmmm lets be optimistic and say it was sent shall we.

    
You have been invited to upload your recipes to Social Recipe! 
<fb:name uid="<%= inviter%>" firstnameonly="false" shownetwork="false"/> wants you to add Social Recipes so you can share recipes together
<fb:req-choice url="http://www.facebook.com/add.php?api_key=916c58dd4fba835cc43389a432fa98d0" label="Add" />
<fb:req-choice url="http://apps.facebook.com/socialrecipe" label="Go to Application" />
    

The fb:req-choice will give the user in the notification the options to add the application or go to it. If you now submit the form

Wrapup

So that’s all, there are a few more things to discover in the application interface but you should have all the basic elements to create some fun apps to share with your friends and their friends. There is certainly not enough error handling in here for a production application but i’ll leave that up to the reader.

Episode 58: How to Make a Generator

Posted about 7 years back at Railscasts

Rails comes with many built-in generators, but what if you want to customize them or make your own? Rails gives you a great way to do this which you will learn in this episode.

Episode 57: Create Model Through Text Field

Posted about 7 years back at Railscasts

Let's say you provide a select menu for setting which category a given product belongs to, but you also want the option of creating a new category by typing the name in a text field. See a great way to do that in this episode.

Good to Great part 2 - Going Full Circle

Posted about 7 years back at work.rowanhick.com

Following on from Good to Great (rules to live by) post. One thing that constantly irks me is the "It's not my job" syndrome. This is most apparent in call centers "Sorry I can't help you with this, let me transfer you to.." [cue another 10min wait time]. We've all been on the receiving end of that one and can attest to how we get annoyed by it. This also manifests itself in development teams and developed systems. The biggest impact you as a IT person (be it developer, analyst, project manager) can make, is take a 'full circle' approach to your interactions both within a team and directly with clients and your developments. What do I mean ? take extra time out of your day when faced with an issue, that's not technically your, or your apps responsibility, to solve it completely for your client or team member. Or if you can't solve it, take ownership in handing it off properly**. The end result is to have the initiator of the request client, user etc, not required to go elsewhere to solve their initial problem. Why would we possibly want to do this, when our time could be better served browsing Digg, checking out Facebook, breaking out the office xbox ? - Client is vastly happier, happy clients pay bills, come back for more work, and refer you/your company - You get credit, and respect within the team and from the client - You will likely gain greater domain knowledge - You will likely see opportunities in your app to further improve it's usability Lets look at some examples: ... as applied to a hosting management Case: Your client forget to renew their domain name The standard approach is to send them an email 'you need to renew your domain name'. The full circle approach is to lookup their domain registry details, tell them where and how to renew their domain name - as so often clients have simply no idea. Takes an extra 15mins of your time but you will be rewarded massively in return. If you're really on the ball you'll pop in your calendar an annual recurring reminder to make sure it gets renewed ahead of time. ... as applied to internal development Case: You stumbled across a bug you think most people won't notice. The likely approach would be to fix the bug you found and case closed. Lets say you made a mistake that caused all historical orders in a system to miss the city off the shipping details. So the first approach is to go 'ooops, shouldn't have done that, let's fix it here and move on'. Taking a full circle approach to it would be to go and contact everyone who was affected by it and let them know what the problem was, how it would present itself, that you found the problem, fixed it, and what would occur now. By being open and honest with them and taking the time out to inform them, will gain trust and appreciation. Suffice to say if the little bug was caused by you and resulted in a minor loss of a few million dollars it might be time to hang up the keyboard and quickly get out of dodge. ... as applied to developing a system Case: An order system ships stuff via a major shipper, tracking number's get emailed to the buyer. ( This one is my personal pet peeve ) How about your system goes away to your shipper(s) requests shipping information, and updates your user. So you don't require your user to go elsewhere to find shipping information. Better yet - when your shipping service says delivered how about an email 24 hrs later "We have been notified your package has been delivered, please click on this link to let us know you've received your package and if you may have any comments you can make them there". That way your system has a true end to end connection with the user from the time the order was placed, till it hits their doorstep. In summary Whenever interacting with someone, take time out to put yourselves in their shoes, and ask yourself was this problem completely solved for me? Take ownership and pride in the solution and you will be rewarded 10x over with fellow team members and clients, willingness to work with you in the future. ___ ** Handing something off properly ? How so ? For the love of god never expect your client to explain everything all over again to another team member. Talk to the client, say you can't personally solve it, however you've given all the details to person x and they will be contacting them, you'll follow up in y timeframe to make sure it's all their issue has been dealt with a-ok.

Sidebar Login Box Recipe (restful_auth)

Posted about 7 years back at zerosum dirt(nap) - Home

One of the nice features of Rick Olson’s excellent restful_authentication plugin is the store_location facility. If you set up a :login_required before filter in your controller, the access_denied method will get called if the user isn’t logged in. The requested URL will be stored in the session and the user will be redirected to the login page. After logging in, redirect_back_or_default is called by SessionController#create, which pulls the location out of the session and redirects the user back to where they intended to go in the first place. Very slick.

However, if you’ve built an application which has a login box in the sidebar (I recommend simple_sidebar), restful_auth has no idea where you were coming from before the post to the create action. So you get redirected to the default (usually ‘/’) specified as a parameter to redirect_back_or_default. This isn’t usually what the user expects. They expect to be returned to the page they were looking at before if they log in from the sidebar. Oh noes!

Here’s a little recipe you can use to provide that…

First, a simple change to authenticated_system.rb in your lib/ directory (this file is created by the authenticated generator). Change the store_location method to this:

def store_location(location = request.request_uri)
  session[:return_to] = location
end

Before this change, store_location took 0 parameters and always set the return_to location to the current request URI. Now, we’ve modified it so it can (optionally) take the location to store as a parameter. If this parameter isn’t supplied, it defaults to the old behavior.

Next, change the create method in your session controller so it checks the HTTP referer. Just add one tiny little line to the top of the method:

def create
  check_referer
  ...
end

Now let’s write that check_referer method (it should be protected, not public):

def check_referer
  referer = request.env['HTTP_REFERER'] || ""
  if referer.match(request.domain) &&
    !referer.match(session_url) &&
    !referer.match(login_url)
      store_location(request.env['HTTP_REFERER'])
  end
end

What this little utility method does is check the HTTP referer. If the referer is from the current domain, and isn’t the session#create URL or ‘/login’ (in my application, there’s a login_path named route), we go ahead and store the location of the referer. This will later be plucked out by redirect_back_or_default. Note that you’ll need to regex match to make sure the referer value isn’t anything you don’t want to store. For instance, if the user navigates to the login page directly and logs in from there, you don’t want to redirect back to the login page on success. You probably want to redirect to the default page, which is whatever you’re specifying as the default with redirect_back_or_default.

So that’s it! An easy way to handle redirects from sidebar login blocks, without muddying the already elegantly designed redirects that happen when access is denied.

If you want to write a functional test for this, try something like this:

def test_redirect_to_referer # for login sidebar
  location = url_for(:controller => 'foo', :action => 'index')
  @request.env['HTTP_REFERER'] = location
  post :create, :login => 'quentin', :password => 'test'
  assert_response :redirect
  assert_redirected_to location
end

Swing Reduction Sauce

Posted about 7 years back at zerosum dirt(nap) - Home

JRuby continues to develop in interesting ways. Earlier today, Zed Shaw (Mongrel’s dad) announced the release of Profligacy 1.0, which you should take a look at if you’ve got a thing for building cross-platform desktop apps in Ruby. It’s pretty cool.

Profligacy is a (wildly extravagant?) library that purports to take the pain out of using Swing components with JRuby. It uses LEL, Layout Expression Language, to bring a sort of wiki-ish flavor to component layout. This is truly bizarre at first glance, but damn cool at the same time. Basically the Ascii art you lay out with LEL translates into a Swing GroupLayout. Ah-wha? Yep. Check out some examples. I told you it was cool.

It also does some other neat stuff like auto-converting procs to Listener interfaces, but I won’t go on about that, as you can go check the release notes instead. If you’ve ever worked in Swing before, you know how painful it can be. Profligacy makes it significantly less bitter to swallow. Now stop reading blogs and go build your first cross-platform Ruby desktop app. You know you want to.

Can’t find Essential Actionscript 3 and in Canada ?

Posted about 7 years back at work.rowanhick.com

For all the Canadians out there wanting this book (at time of writing) my order from chapters.indigo.ca just shipped, so there must be a few of them in stock - go get your order in quick ! Whereas amazon.ca is currently out of stock, and chapters two weeks ago were out of stock, and O'Reilly is/was of stock when I tried a couple of days ago to order.

Redesign RubyForge?

Posted about 7 years back at Synthesis

With the whole RailsForge survey thing, it’s also come out that a lot of people hate the current incarnation of RubyForge.

Some are suggesting to redesign RubyForge, which is a good idea. Re-think RubyForge and give it some 37signals-style, UI-first design love. I haven’t talked with any RubyForge maintainers yet, but I’d be interested in helping out.

By the way, I totally appreciate RubyForge, and I’m so thankful to the people who run it and donate their time and money to keep it going.

RubyForge pet peeves

What is it about RubyForge that’s so annoying? For me, when I arrive at a project page on RubyForge, I’m greeted with a plethora of “stuff” cluttering the page, that’s either un-used or not really telling me anything. I often leave more confused, and I’m an experienced developer. Imagine how a beginner feels! I then look elsewhere hoping someone has written a helpful tutorial on their blog.

RubyForge-specific complaints from the RailsForge survey responses:

  • Most of the features go unused.
  • Biggest problem is the UI.
  • It’s very difficult to do the things the majority of people want – 1. learn about a project, 2. download and install it, 3. learn how to use it.
  • 90% of the project page has nothing to do with any of this and it just adds clutter.
Suggestions:
  • Let project maintainers selectively turn on the features they want.
  • 80% want to learn. 20% want to contribute. Refocus the UI to reflect this.

A choice of databases, or PostGres vs SQLite

Posted about 7 years back at Saaien Tist

When managing data in bioinformatics, one of the main tools you use is databases to store the stuff. In many cases, flatfiles are sufficient, but sometimes you need the flexibility and reliance of a nice relational database. (Note: Excel is not a database). Actually, in quite a few cases it does make sense to use flat text files instead of databases, but that's the subject of another post.

As there is a plethora of RDBMS (relational database management systems) available, the first thing you have to ask yourself is what system to use. There's mysql, oracle and others. But for the purpose of this post, I'll focus on PostGreSQL (http://www.postgresql.org) and SQLite3 (http://www.sqlite.org), because that's what I normally use.

For the big projects with a lot of data, I normally use postgres, while smaller projects are typically served by sqlite databases. Both of these however have their good and bad things. Choice depends on a number of factors:

Installation. The main difference between postgres and sqlite3 is that one is based on running a server and the other is not. To be able to use postgres, you'll have to install the software, create a postgres user on your system and have that user start the postgres server. Next thing to do is create a user within the database system and grant it the right permissions (e.g. to create other users or databases). That's a bit more complex than installing sqlite3.

Database creation. To create a new database in PostGreSQL, you issue a createdb MyDatabase. This will add the database to your RDBMS. You can get into the database with psql MyDatabase. In contrast, sqlite3 databases are just text-files like any other. Create a new databasebase by calling it: sqlite3 my_database.s3db. This will create a file in your present directory called my_database.s3db. Note: you don't have to use the .s3db extension, but that makes it recognizable for the SQLiteAdmin tool (see below).

Use. Once you've got your databases created, their use is really similar. On the command line, use the psql or sqlite3 command. On Windows, you can e.g. pgAdmin and SQLiteAdmin.

I must say that psql is a bit easier on day-to-day typing, because it's got tab-completion, which I haven't been able to activate yet in sqlite3. I should spend some time one day to configure sqlite exactly like I want it to.

Locking. This is what bit me in the ankles with sqlite3... According to the website: "SQLite version 2.8 allowed multiple simultaneous readers or a single writer but not both. SQLite version 3.0 allows one process to begin writing the database while other processes continue to read. The writer must still obtain an exclusive lock on the database for a brief interval in order to commit its changes, but the exclusive lock is no longer required for the entire write operation." Indeed, I could load hundreds/thousands of records while simultaneously querying the database. But at what looked like a random moment, after hours of loading, it would choke, leaving me with a partially populated database. Any small child can tell you that this is not what you want to happen. I've never encountered something like that in postgres, and probably never will.

Backup. Because sqlite if file-based, making a backup of a database is nothing more than taking a copy and moving that copy to another hard disk or burning it on a CD-ROM. To make backups of postgres databases, we typically use pg_dump, which dumps the contents (and if you want the schema as well) of a database in a big-ass text file. So the backup is outside of the RDBMS itself.

So both RDBMS have their advantages and disadvantages. For huge projects, I tend to use PostGres, although I've started to use sqlite more and more lately. If only it wouldn't choke on the locking...

Episode 56: The Logger

Posted about 7 years back at Railscasts

Learn all about the logger in this episode. See how to send messages to the log and how to customize it.