Hobo 0.5.2 released

Posted over 7 years back at The Hobo Blog

OK 0.5.2 is now available, both in the svn trunk and as a gem

* hobo-0.5.2.gem

* Changelog

The important breaking change is that the customisation hooks in your hobo_model_controllers no longer work. e.g. if you have

def create_response
  redirect_to "..."
end

That will no longer work. The equivalent is now:

def create
  hobo_create :html_response => proc { 
  redirect_to "..."
  }
end

If you have done a lot of stuff with those hooks, you might want to wait until tomorrow before grabbing 0.5.2. I’m hoping to put a decent chunk of time into catching up with the documentation tomorrow.

Enjoy!

Hobo 0.5.2 released

Posted over 7 years back at The Hobo Blog

OK 0.5.2 is now available, both in the svn trunk and as a gem

* hobo-0.5.2.gem

* Changelog

The important breaking change is that the customisation hooks in your hobo_model_controllers no longer work. e.g. if you have

def create_response
  redirect_to "..."
end

That will no longer work. The equivalent is now:

def create
  hobo_create :html_response => proc { 
  redirect_to "..."
  }
end

If you have done a lot of stuff with those hooks, you might want to wait until tomorrow before grabbing 0.5.2. I’m hoping to put a decent chunk of time into catching up with the documentation tomorrow.

Enjoy!

Episode 21: Super Simple Authentication

Posted over 7 years back at Railscasts

The final piece of the administration puzzle: authentication. There are many different approaches which is why I saved this step for last. This episode will cover a few techniques including the simple solution used for this site.

Episode 20: Restricting Access

Posted over 7 years back at Railscasts

In this second part of the series on administration, you will learn how to lock down the site to keep the public from accessing the administration features.

REST vs AutoComplete

Posted over 7 years back at zerosum dirt(nap) - Home

After struggling with it probably a little bit more than I should have, I’ve totally come around to the REST way of looking at things. It seems clean and nice, and I’m 100% behind it. That said, I’ve found that once you adopt the mentality, things that don’t gel with it start to stick out like the guy in the fur coat at the PETA meeting.

Case in point: the ever-popular Rails auto_complete_field helper. To back up my accusation here, let’s walk through an example scenario with a REST app, and examine the issues that arise…

Imagine a really simple web application which consists of a collection of Articles. The only user interaction is that guests can comment on articles and tag them with keywords. So in a RESTful world, we have an Articles resource, and a Tags resource. Tags are also nested on Articles such that when you ask for /articles/:article_id/tags you’re asking for the subset of tags which have been applied to article_id. We might present these as a tab you can click on when viewing an article resource in a browser.

Now let’s say that there’s a form somewhere (maybe the #new action on an ArticleCommentsController), which can be used to apply new tags to a given article. There’s an auto_complete_field in that form, that will suggest tags to apply to an article, based on the tags that have already been applied. Dumb example, I know, but hear me out.

So, given this scenario, it would make sense to point our auto_complete_field helper at article_tags_path(article_id). This is about as RESTful as it gets — we’ve become a client of our own API. Nice! Our #index action on ArticleTagsController, in a perfect world, would do a couple things above and beyond just displaying the pretty HTML list of tags on the article:

  • It would have a respond_to block. if a browser client is asking for XHTML, we probably want to render a nice paginated view of the relevant tags, ordered by weight, with a nice layout. On the other hand, if we’re asking for XML or JSON, it just returns no-frills data in the appropriate format.
  • it would optionally check for a filter query string. perhaps we set params[:filter] = :attribute_name and then make the actual string filter value (the thing we’re trying to help auto-complete) available via params[:attribute_name]. in the case of our tags example, we’d end up with a params hash that has filter=name and name=zer (or some other text string). If a filter parameter is found, we’d filter the results by the pattern specified. This could all be wrapped up in a nice little helper.

This proposal raises a couple issues. First, the auto_complete_field helper as it exists today expects us to return some HTML from the AJAX call that we can plug into an appropriately named DIV. This is simple and works well, but if we’re really consuming our own web service, we should probably be talking to it via JSON or XML and have a JavaScript handler to process the results, converting them into list format or whatever it is that we desire. But I have to admit that this convenient hack works nicely.

More importantly, if the auto-complete filter is being applied to the collection of resources, that means it maps to the #index action by way of a GET request. So our parameters end up encoded in the URL with a query string like ?filter=name&name=zer. Obviously, GET params are a little more limiting than POST params in terms of length and (perceived) security. But for simple filter strings that take a short sequence of characters, I don’t see this as being an issue. And this is always the case when doing an auto-complete, as far as I can figure.

Note that the auto_complete_field JavaScript helper doesn’t actually allow you to specify an HTTP verb other than POST, although that’s easy enough to change. Surely the core team would accept the one-line fix required to support a :method => :get option if it’s the “right thing to do”. The real problem is more of a philosophical one: are we, as a community, really going to embrace REST 100% or is it acceptable to add small bits of cruftiness in places where it seems inconvenient?

A perfectly legitimate half-way solution is to define a standard #autocomplete action for any resource you want to use in this manner. The advantage to this is that you can easily use a POST against a custom :collection method on the resource. That way, we continue to work inside the existing box of conventions. But I ask you, is that really RESTful?! What do you think?

Update

Posted over 7 years back at The Hobo Blog

You may have noticed I’ve been a little quiet of late—both on the blog and the forums. It’s end-of-project syndrome. We’re just coming to the end of the final delivery in the first major client project built on top of Hobo. Deadlines are hard work!

Here’s a bit of an update about what’s been going on with Hobo recently.

First-up, Hobo 0.5.2 is due to go out. I’ll make a commitment, because I’ve found that’s a good way to ensure it actually happens - I’ll get 0.5.2 out by the end of this week. In this version are a bunch of fixes as always, the new mechanism to customise controller behaviour, and a nice extension to ActiveRecord which came out the recent stuff on scope_out.

If you followed the comments on that post you probably saw the link to neotrivium from kaspar (thanks!). A very nice development of the ideas in scope_out but lacking some scope_out features such as cacheing of scoped associations. All this inspired me to see if I could take the idea a bit further, and you can see how I got on in Hobo 0.5.2. By way of a sneak preview you can do:

Post.comments.published

And get back a full AR association proxy, which you can either treat as an array, or use to do further finding, counting etc. All this for the bargain price of:

def_scope :published, :conditions => "published = 1"

In the Comments class. You can also do things like:

Comments.published.find(...)
Comments.published.boring.count

Handy! (the SQL fragment to identify a boring comment is left as an exercise for the reader)

Let’s see, what else..?

I’m waaay behind on the forums, so a big apology to anyone who’s looking for help and hasn’t yet received any. I’ll get there :-)

On the subject of community - I was very happily surprised to get this mail yesterday:

Hey Tom,

We started a #hobo channel on freenode for hobo users. Please come visit us irc.freenode.net #hobo and help support your freenode hobo community users :) We are supporting eachother right now and would like to see you come through.

UncleD @ http://rubyonrailsblog.com

Awesome! I’m afraid to say I haven’t shown my face as yet (did I mention I’ve had a deadline on?) I definitely will stop by from time to time, but I must warn you I’m not a great IRCer.

And saving the best news for last–a little tentative at this stage, but potentially a big deal. We at HoboTech are close to making our first hire. I’m REALLY looking forward to the prospect of NOT BEING RUN COMPLETELY OFF MY FEET ALL DAY EVERY DAY!!! Sorry just venting some frustration :-) If this works out it’s going to have a big impact on the time I have to focus on the many aspects of Hobo (documentation!), so cross your fingers.

Catch you again later this week with the “0.5.2 Released” post :-)

Update

Posted over 7 years back at The Hobo Blog

You may have noticed I’ve been a little quiet of late—both on the blog and the forums. It’s end-of-project syndrome. We’re just coming to the end of the final delivery in the first major client project built on top of Hobo. Deadlines are hard work!

Here’s a bit of an update about what’s been going on with Hobo recently.

First-up, Hobo 0.5.2 is due to go out. I’ll make a commitment, because I’ve found that’s a good way to ensure it actually happens - I’ll get 0.5.2 out by the end of this week. In this version are a bunch of fixes as always, the new mechanism to customise controller behaviour, and a nice extension to ActiveRecord which came out the recent stuff on scope_out.

If you followed the comments on that post you probably saw the link to neotrivium from kaspar (thanks!). A very nice development of the ideas in scope_out but lacking some scope_out features such as cacheing of scoped associations. All this inspired me to see if I could take the idea a bit further, and you can see how I got on in Hobo 0.5.2. By way of a sneak preview you can do:

Post.comments.published

And get back a full AR association proxy, which you can either treat as an array, or use to do further finding, counting etc. All this for the bargain price of:

def_scope :published, :conditions => "published = 1"

In the Comments class. You can also do things like:

Comments.published.find(...)
Comments.published.boring.count

Handy! (the SQL fragment to identify a boring comment is left as an exercise for the reader)

Let’s see, what else..?

I’m waaay behind on the forums, so a big apology to anyone who’s looking for help and hasn’t yet received any. I’ll get there :-)

On the subject of community - I was very happily surprised to get this mail yesterday:

Hey Tom,

We started a #hobo channel on freenode for hobo users. Please come visit us irc.freenode.net #hobo and help support your freenode hobo community users :) We are supporting eachother right now and would like to see you come through.

UncleD @ http://rubyonrailsblog.com

Awesome! I’m afraid to say I haven’t shown my face as yet (did I mention I’ve had a deadline on?) I definitely will stop by from time to time, but I must warn you I’m not a great IRCer.

And saving the best news for last–a little tentative at this stage, but potentially a big deal. We at HoboTech are close to making our first hire. I’m REALLY looking forward to the prospect of NOT BEING RUN COMPLETELY OFF MY FEET ALL DAY EVERY DAY!!! Sorry just venting some frustration :-) If this works out it’s going to have a big impact on the time I have to focus on the many aspects of Hobo (documentation!), so cross your fingers.

Catch you again later this week with the “0.5.2 Released” post :-)

NH Ruby UG Lightening Talks Tonight!

Posted over 7 years back at zerosum dirt(nap) - Home

Join us tonight at the Portsmouth Public Library for a bunch of lightening talks on Ruby-related subjects. I’ll be speaking on REST, Rake, and RMagick (if I can manage to put together a preso and demo app in time). Scott will be presenting a demo of his Rails-based ProgressPuppy task manager. And there will be other stuff, too.

Note that we’re still doing the meeting, despite aftereffects of the nasty coastal flooding. Hardcore, eh? As always, more information (including directions) can be found at the NHRuby Wiki.

Lesson #5

Posted over 7 years back at Ryan Tomayko's Writings

Ian Bicking in Python’s Makefile:

An example of this is buildutils. This package is, sadly, unmaintained; in part I'm writing this blog post to encourage people to steal/remake/repair the kind of functionality buildutils provides, and provide entirely new functionality.

Buildutils is one of two Python projects I really screwed up on in my maintainership duties. The other is a documentation utility called Pudge. On both, I failed to heed the wisdom of Lesson #5:

  1. When you lose interest in a program, your last duty to it is to hand it off to a competent successor.

I never lost interest in either of these but had to stop working on them due to other commitments. Both are conceptually solid and somewhere near 50% through their implementation cycle. Each would fill a big hole in today’s Python toolkit (IMO). Neither are being maintained. No mailing lists, no new releases, nothing.

Projects die all the time due to lack of interest and value. That’s normal and healthy and a big part of the F/OSS ecosystem working its magic. What’s not normal and healthy is when projects that people are interested in die. That’s bad maintainership.

I'm sorry.

One thing about Lesson #5 that isn’t immediately obvious is that it’s not okay to just leave a dead project out there. You must find a maintainer or you must die all the way. At the very least, you have to state clearly that the project is no longer being maintained and that people are encouraged to start new projects that duplicate the dead project’s functionality.

The first issue here is that people will find your project and believe that someone is already working on the problem and so will not want to duplicate the effort. The second issue is that leaving dead bodies on the field decreases morale. Who wants to start up another documentation project when there’s five half attempts already out there? If the previous projects couldn’t get it done, there must be something troubling (like dragons) about the problem itself. Nope, the maintainers probably just got tied up on something else.

All that being said, I'd like to humbly request that anyone who has any interest at all in these projects and would be interested in taking over maintainer duties, please leave a comment or contact me via email.

If the projects are resurrected, I'd prefer that the subversion repositories and project sites be moved to a new home but I can host this stuff for as long as needed. I've had multiple requests (the frequency of which seems to be increasing) for the mailing lists to come back but I can’t host those anymore so finding a new discussion area would be the first order of business.

In the meantime, I'm going to add text to the project sites noting their status (as being dead) with a link to this article and an invitation for gratuitous code theft.

One last thing… If you've never maintained a F/OSS project before but are interested, either of these projects would be a great place to start. They’re both fun if you’re into tooling, neither would require a whole lot of time, and there’s already small communities of really smart folks who use the projects and would contribute.

Episode 19: Where Administration Goes

Posted over 7 years back at Railscasts

This is the first part of a three part series on building an administration system. Many developers choose to separate it completely from the public side of the site by placing it into an admin subdirectory. This episode will show you an alternative and often better solution.

Rails and Scaling with Multiple Databases

Posted over 7 years back at Ryan Tomayko's Writings

Joe points out the following from this interview with Twitter’s Alex Payne:

The problem is that more instances of Rails (running as part of a Mongrel cluster, in our case) means more requests to your database. At this point in time there’s no facility in Rails to talk to more than one database at a time.

(David and Rafe weigh in also. Good stuff)

We've run against multiple (five now) separate PostgreSQL servers for a long time now. To be clear, that’s five separate “databases” in the sense that PostgreSQL uses the term. Not a replicating / mirror setup – separate databases with different data but with similar structure.

Each of our clients gets a schema (again, in the postgres sense of the word) on one of the database boxes. There are multiple client schemas per database. When a box becomes over utilized we get another box and move schemas around.

Sidebar: this whole schema-per-client thing might seem like overkill but when we say client, we mean, a company that processes health claims for 50 to 1000 self-funded employers. Each client is fairly massive in the volume of data they load into the system and our entire market consists of maybe 400 total prospects. Those 400 companies process something like 60% of the covered US population’s health claims every year.

In addition to the client databases, we have a master-slave balanced and replicated shared schema that stores user account information and other data applicable to all clients.

We run multiple fastcgi dispatchers on two severely underutilized web boxes. Our database load dwarfs our web load as we’re doing live data-warehouse style queries, which are insanely intensive when compared to generating the HTML or PDFs to display the results.

When an HTTP request comes in, it can go to any of the fastcgi dispatchers. We practice shared nothing. We establish, from the shared schema, which client the user belongs to and can then determine the server and schema that houses their data. Each fastcgi dispatcher has a pool of connections: one dispatcher has one connection to each postgres server. We scope the connection in using a set of Rails extensions similar to ActiveRecord::Base::with_scope at the beginning of the request.

This situation is surely very different from Twitter’s but I hope it shows the difference between the statement made by Alex, “there’s no facility in Rails to talk to more than one database at a time,” and the significantly more problematic, “you cannot talk to more than one database at a time.” While the former may be true (and I’ll argue in a moment that it’s not), the latter is clearly false.

Talking about this on the level of having “multi-database connectivity” come out of the box in Rails is simplifying the problem to an unreasonable level, I think. Would the multi-database connectivity features meet my needs or Twitter’s? They are different problems with minor overlap and we’re talking about the minor overlapping piece like it’s the biggest part of the problem.

Most of the time spent getting our setup running was in the conceptual and data wrangling phases. The amount of time it took to implement the multi-database connectivity was negligible compared to the amount of time it took to devise a method of splitting things out at the data level. When all was said and done, the Ruby/Rails related bits were implemented in no more than 40-50 lines of code.

In my case, ActiveRecord provided exactly the right level of functionality. I can have multiple database connections established and write code to manage when each should be used. Control of which connection is used is managed at the model level. Connections cascade up inheritance chains and I can specify that one model use the connection specified on another model using a simple delegate statement:

class A < ActiveRecord::Base
end

class B < ActiveRecord::Base
   class << self
     delegate :connection, :to => A
   end
end

Changing A’s connection changes B’s without effecting the connection used by any other model. We have a simple macro (uses_connection_of) that brings this down to a one liner for each top level model class:

class C < ActiveRecord::Base
  uses_connection_of B
end

This is only the tip of the framework level customizations we've made to Rails over almost two years of development. In most cases, I find the base functionality well balanced for the general (80%) case. We expect to write additional framework code when we get into special case territory, which our multi-database/schema setup clearly is, and which Twitter’s seems to also be.

When I consider what contributed to the unraveling of J2EE, one thing that stands out is that it tried to do too much. The promise was that of infinite scalability based on tooling, which assumes that designing scalable systems is a general case problem. I now firmly believe that this is flawed reasoning. Frameworks don’t solve scalability problems, design solves scalability problems.

I picked up a word from Joe a few years back and find myself using it a lot: “friction.” When referring to framework and tooling, “friction” is a (subjective) measure of how much the tooling gets in your way when trying to solve a specific-case problem. I've come to evaluate frameworks based on two rough metrics: how far the framework goes in solving the general case problem out of the box and how little friction the framework creates when you have to solve the specific-case problem yourself. When a framework finds a balance between these two areas, we call it “well designed.”

Measured along these lines, there are portions of Rails that have a less than perfect balance but I don’t think multi-database connectivity is one of them. It seems to me that moving too far in one direction on this would cause lots of friction for moving in other directions. There just doesn’t seem like there’s a lot of general case to solve here when you dig into the details.

Bottom line for me is that Twitter’s scaling and multi-database connection issues seem to be just that: Twitter’s issues. David’s response seems to indicate that he believes Rails could probably do more here but how far could framework level support really go and how much friction would be created?

Episode 18: Looping Through Flash

Posted over 7 years back at Railscasts

Displaying flash messages in the layout can be a pain at times. In this episode you will learn an easy way to display any kind of flash message by looping through the hash.

Episode 17: HABTM Checkboxes

Posted over 7 years back at Railscasts

It is often asked: how do I create a list of checkboxes for managing a HABTM association? Ask no more because this episode will show you how to do exactly that.

Productivity Enhancers

Posted over 7 years back at work.rowanhick.com

Tannoy nearfield active monitors. Let's face it churning through documentation or code, can sometimes be mind numbing, no matter how much delegation you do you still have to deal with it and the only way to keep your mind in check, and sometimes to speed things along is to be bathed in your favourite music. Now I have a penchant for all things musical, whilst I don't spend 40k on audio gear at home, I also can't stomach the $50 computer speakers. A previous fan of Klipsch systems I decided it was time to step up the game. Enter these bad boys. "Nearfield active monitors" = speakers designed to be on a studio recording desk, up close to your ears versus across the living room floor. As they're monitors they're designed to be very neutral and flat sounding, allowing the detail of the music to present itself, not be coloured like most residential systems. So you won't have bass or mid range heavy music. To my ears, at low volumes everything is very crisp, tonnes of detail I've picked things I've only heard with Sennheisers on my head. All this comes at a cost, these clock in at $800 retail CAD, before taxes, if you know where to look. However eBay came to the rescue and I found a pair and won the auction for under half that, now they're gracing my desk getting burned in. Nice. (with a capital N). Not Pants.

Productivity enhancers

Posted over 7 years back at work.rowanhick.com

Tannoy nearfield active monitors. Let's face it churning through documentation or code, can sometimes be mind numbing, no matter how much delegation you do you still have to deal with it and the only way to keep your mind in check, and sometimes to speed things along is to be bathed in your favourite music. Now I have a penchant for all things musical, whilst I don't spend 40k on audio gear at home, I also can't stomach the $50 computer speakers. A previous fan of Klipsch systems I decided it was time to step up the game. Enter these bad boys. "Nearfield active monitors" = speakers designed to be on a studio recording desk, up close to your ears versus across the living room floor. As they're monitors they're designed to be very neutral and flat sounding, allowing the detail of the music to present itself, not be coloured like most residential systems. So you won't have bass or mid range heavy music. To my ears, at low volumes everything is very crisp, tonnes of detail I've picked things I've only heard with Sennheisers on my head. All this comes at a cost, these clock in at $800 retail CAD, before taxes, if you know where to look. However eBay came to the rescue and I found a pair and won the auction for under half that, now they're gracing my desk getting burned in. Nice. (with a capital N). Not Pants.