Breaking the Silence

Posted over 7 years back at The Hobo Blog

January 26th? 6 weeks ago!?! Is it really that long since I last posted? Wow. I do apologise for keeping you all hanging for so long. Really.

The funny thing is I guess it hadn’t really sunk in that anyone would be that bothered. I mean, I do see the stats – we’re getting a fairly steady 600ish visits per day – but what does that mean? It’s just a number. Maybe a moderately large number on the scale of all the blogs out there, but there’s nothing that powerful about a number, not compared to, say, the human voice. And then suddenly you have a voice – the previous post now has some fairly heart-felt comments that carry more weight than web stats ever could.

To be honest it’s a little bit overwhelming for me. I’m just some guy with zero prior standing in the Ruby community who wrote a Rails plugin. I stop blogging and someone actually gives a hoot? Really? People are annoyed that I stopped?! Woah!

But OK - if that’s how things are, I guess I better step up. A change is definitely needed. We need to solve a problem nicely articulated by Michiel:

Currently, Hobo feels like a closed tower. Updates are announced without a timetable or a feature list. So no-one can guess what’s coming next or when it will arrive. This is obviously frustrating because none of us can really work or play with Hobo like this.

I would advise the Hobo team to blog a bit more about various features and use-cases instead of putting the focus on documentation and new releases with new features.

A good way to come down from the tower would be to tell you a little about the “Hobo team”. Um. There isn’t one. It’s just me. Well, there’s one other guy deeply involved - my good friend Nigel Powell. He’s chief evangelist, cheerleader and fellow Hobo visionary. Nigel has contributed a lot to the vision for Hobo (Oh, and he came up with the name BTW). Nigel shows up on the blog as Red, which is his nickname from his blog The Red Ferret Journal (which is a fun read if you’re into tech trivia). So when I say there is no team, I don’t mean to disregard all that Nigel has done for the project, but Nigel is not a coder. So the stuff you guys are really interested in – Hobo itself, documentation – there is no team for all that. Just me, and of course – you!

OK, that’s who we are, now a bit about what we’re doing with Hobo. Hobo was ‘extracted’ from an app I was building, much as Rails was extracted from Basecamp. I was just doing what programmers do - trying to keep the abstraction level in my app very high so I didn’t have to repeat myself, and could take a very agile approach to evolving the app. OK DRYML was a bit of an indulgence, but I’d built something similar before a couple of times (in Perl, circa 1994, and later in Python), and I was able to knock up a working prototype in a couple of days. When DRYML was working I looked at what I had, and thought hey, this is cool, I could release this. I showed what I had to Nigel, and he pounced on it.

Things started to move pretty quickly from there, and as our enthusiasm grew we decided to form HoboTech Ltd. A web-development / consulting firm that uses Hobo as its not-so-secret weapon. We’re just nearing the completion of our first major project and are very happy about the fact that the use of Hobo has been an enormous success.

So that’s what we’re doing - commercial Rails development projects, taking advantage of Hobo, and effectively funding the development of Hobo at the same time. It’s kind of fascinating how this approach is so much better than seeking venture funding. Having to deal with the idiosyncrasies of clients’ projects sometimes feels like a pain, but it keeps Hobo real.

Of course it can be frustrating that I don’t have all the time I’d like to throw into pure Hobo, and it’s inevitable that doesn’t-pay-the-bills work like documenting Hobo is the first to suffer. But overall, this real-world work is making Hobo way better than it otherwise would be. Agreed, good code with bad docs is frustrating, but good docs for bad code is just plain useless.

That’s all well and good, but what about Hobo? What’s going on with the 0.5 release, I hear you ask? The reason for the hold-up is that we’ve decided to do a kind of a second launch. We launched Hobo with a screencast that gave just a glimpse of what Hobo does, a fairly buggy codebase, and a web-site that was broken in the world’s dominant browser. And then, to our total amazement, we had 20,000 visits in January. We’d like to do it again properly. Hobo has now been used in a production quality app other than the app it was spun out of, which is a hugely important milestone. We have two new screencasts, and together the three do a vastly better job of showing what Hobo is about. We have a “mini manual” nearing completion, which though compact should be very useful. And we’ve nearly finished the shiny new web-site. To be honest, in terms of the codebase, Hobo 0.5 won’t be that far from Hobo 0.4.3, but there is one new feature in there that I’m really excited about (which I’ll tell you all about in a post soon).

We honestly thought all this would be done weeks ago, but we got busy with HoboTech work, and some easy things turned out not to be (gasp!).

But enough is enough - all these things will land on the Interwebs no later than this Wednesday 14th (that’s GMT BTW - just in case it goes to the wire!)

Hopefully this post has set the tone for a more open and conversational future. I have some very interesting ideas that I haven’t even begun to implement yet, and I think now I’ll blog about them first, to see what people think. And I promise to post more often!

Oh and one last thing - can I just state the obvious and remind everyone that if you are trying to use Hobo, you are trying to use an early beta. If that’s what you want to do, I am stupendously grateful to you, but please expect the documentation to be lamentable! Please expect the ride to be bumpy! I will try very hard to make things smoother for you. It will take a little time! Hopefully we’ll do it together.

Breaking the Silence

Posted over 7 years back at The Hobo Blog

January 26th? 6 weeks ago!?! Is it really that long since I last posted? Wow. I do apologise for keeping you all hanging for so long. Really.

The funny thing is I guess it hadn’t really sunk in that anyone would be that bothered. I mean, I do see the stats – we’re getting a fairly steady 600ish visits per day – but what does that mean? It’s just a number. Maybe a moderately large number on the scale of all the blogs out there, but there’s nothing that powerful about a number, not compared to, say, the human voice. And then suddenly you have a voice – the previous post now has some fairly heart-felt comments that carry more weight than web stats ever could.

To be honest it’s a little bit overwhelming for me. I’m just some guy with zero prior standing in the Ruby community who wrote a Rails plugin. I stop blogging and someone actually gives a hoot? Really? People are annoyed that I stopped?! Woah!

But OK - if that’s how things are, I guess I better step up. A change is definitely needed. We need to solve a problem nicely articulated by Michiel:

Currently, Hobo feels like a closed tower. Updates are announced without a timetable or a feature list. So no-one can guess what’s coming next or when it will arrive. This is obviously frustrating because none of us can really work or play with Hobo like this.

I would advise the Hobo team to blog a bit more about various features and use-cases instead of putting the focus on documentation and new releases with new features.

A good way to come down from the tower would be to tell you a little about the “Hobo team”. Um. There isn’t one. It’s just me. Well, there’s one other guy deeply involved - my good friend Nigel Powell. He’s chief evangelist, cheerleader and fellow Hobo visionary. Nigel has contributed a lot to the vision for Hobo (Oh, and he came up with the name BTW). Nigel shows up on the blog as Red, which is his nickname from his blog The Red Ferret Journal (which is a fun read if you’re into tech trivia). So when I say there is no team, I don’t mean to disregard all that Nigel has done for the project, but Nigel is not a coder. So the stuff you guys are really interested in – Hobo itself, documentation – there is no team for all that. Just me, and of course – you!

OK, that’s who we are, now a bit about what we’re doing with Hobo. Hobo was ‘extracted’ from an app I was building, much as Rails was extracted from Basecamp. I was just doing what programmers do - trying to keep the abstraction level in my app very high so I didn’t have to repeat myself, and could take a very agile approach to evolving the app. OK DRYML was a bit of an indulgence, but I’d built something similar before a couple of times (in Perl, circa 1994, and later in Python), and I was able to knock up a working prototype in a couple of days. When DRYML was working I looked at what I had, and thought hey, this is cool, I could release this. I showed what I had to Nigel, and he pounced on it.

Things started to move pretty quickly from there, and as our enthusiasm grew we decided to form HoboTech Ltd. A web-development / consulting firm that uses Hobo as its not-so-secret weapon. We’re just nearing the completion of our first major project and are very happy about the fact that the use of Hobo has been an enormous success.

So that’s what we’re doing - commercial Rails development projects, taking advantage of Hobo, and effectively funding the development of Hobo at the same time. It’s kind of fascinating how this approach is so much better than seeking venture funding. Having to deal with the idiosyncrasies of clients’ projects sometimes feels like a pain, but it keeps Hobo real.

Of course it can be frustrating that I don’t have all the time I’d like to throw into pure Hobo, and it’s inevitable that doesn’t-pay-the-bills work like documenting Hobo is the first to suffer. But overall, this real-world work is making Hobo way better than it otherwise would be. Agreed, good code with bad docs is frustrating, but good docs for bad code is just plain useless.

That’s all well and good, but what about Hobo? What’s going on with the 0.5 release, I hear you ask? The reason for the hold-up is that we’ve decided to do a kind of a second launch. We launched Hobo with a screencast that gave just a glimpse of what Hobo does, a fairly buggy codebase, and a web-site that was broken in the world’s dominant browser. And then, to our total amazement, we had 20,000 visits in January. We’d like to do it again properly. Hobo has now been used in a production quality app other than the app it was spun out of, which is a hugely important milestone. We have two new screencasts, and together the three do a vastly better job of showing what Hobo is about. We have a “mini manual” nearing completion, which though compact should be very useful. And we’ve nearly finished the shiny new web-site. To be honest, in terms of the codebase, Hobo 0.5 won’t be that far from Hobo 0.4.3, but there is one new feature in there that I’m really excited about (which I’ll tell you all about in a post soon).

We honestly thought all this would be done weeks ago, but we got busy with HoboTech work, and some easy things turned out not to be (gasp!).

But enough is enough - all these things will land on the Interwebs no later than this Wednesday 14th (that’s GMT BTW - just in case it goes to the wire!)

Hopefully this post has set the tone for a more open and conversational future. I have some very interesting ideas that I haven’t even begun to implement yet, and I think now I’ll blog about them first, to see what people think. And I promise to post more often!

Oh and one last thing - can I just state the obvious and remind everyone that if you are trying to use Hobo, you are trying to use an early beta. If that’s what you want to do, I am stupendously grateful to you, but please expect the documentation to be lamentable! Please expect the ride to be bumpy! I will try very hard to make things smoother for you. It will take a little time! Hopefully we’ll do it together.

The Camping Episode II - Ruby on Rails Podcast

Posted over 7 years back at Ruby on Rails Podcast

After a month, Camping Episode II!

  • Chris van Pelt on his cropper and presenter apps.
  • Manfred Stienstra talks about his HTTP authentication library for Camping and also UTF-8 encoding in “that other web framework.”

Switch & Solo Cards with Website Payments Pro UK (Payflow) and XMLPay

Posted over 7 years back at Cody Fauser

While implementing support for PayPal Website Payments Pro UK in ActiveMerchant I ran across a small problem while processing the UK cards Switch & Solo.

Switch & Solo cards are a special case when processing credit cards, as they need some additional information. The additional information is either the card issue number or start date, as printed on the card. The problem is that the XMLPay guide provided by PayPal doesn't exactly specify where to put this information. If you dig around in the schema it appears as though you can place the CardIssue or CardStart element within a Level3Invoice element, however that didn't seem to work, and always returned the error "Field format error: CARDSTART or CARDISSUE must be present"

After some trial and error, and a helping hand in the right direction from Todd Sieber at PayPal, I was finally able to successfully process the Switch & Solo cards using the ExtData elements within the Card element of the XML request:

1
2
3
4
5
6
7
8
9
10

<Card>
  <CardType>Switch</CardType>
  <CardNum>5641820000000005</CardNum>
  <ExpDate>200801</ExpDate>
  <NameOnCard>Cody Fauser</NameOnCard>
  <CVNum>123</CVNum>
  <ExtData Name='CardIssue' Value='01'/>
  <ExtData Name='CardStart' Value='0299'/>
</Card>

You only need either the CardIssue or CardStart element to be present, but for illustrative purposes I included them both.

One other thing to note is that XMLPay API will no longer be updated by PayPal. So if you are writing new code then you might as well just stick to the Name-Value pair API.

Episode 3: Find Through Association

Posted over 7 years back at Railscasts

No need to pass foreign keys in find conditions, just do the find through a has_many association.

DEISA Sessions (Distributed European Infrastructure for Supercomputing Applications)

Posted over 7 years back at Spejman On Rails

This week I'm taking DEISA training sessions in Barcelona, they are teaching us how to submit tasks into most powerful supercomputers of Europe (DEISA Architecture).

Within this supercomputers we can meet Marenostrum, nowadays is the most powerful supercomputer in Europe and the World's fifth most powerful as you can see in www.top500.org. It's made of 10.240 CPUs Power PCs, 20 TeraBytes of RAM and 370 TeraBytes of disk. It's placed in a old chapel which gives it a special feeling.

Today I've been lucky and I could visit inside Marenostrum as you can see in this photos:

Marenostrum

Marenostrum

Marenostrum Core

More photos: http://www.flickr.com/photos/spejman/tags/marenostrum/

Episode 2: Dynamic find_by Methods

Posted over 7 years back at Railscasts

Shorten simple finds considerably and improve readability by using the dynamic find_all_by and find_by methods.

Tweaking The Rails Logger

Posted over 7 years back at zerosum dirt(nap) - Home

The Ruby Logger is a simple but pretty flexible tool — hopefully you’re already using it. If not, you should be. The default logger message format is pretty barebones though, so I thought I’d take a few minutes to talk about how to make it more useful by monkey patching format_message.

First, make sure to check out TopFunky’s ‘Hodel 3000 compliant logger’ article, posted a couple months ago. Geoffrey’s syslog-friendly modification works great when running in production mode. Customized log messages are just as important in the development environment, where we’ve made our own simple modifications to include the name and line number of the file:

class Logger
  def format_message(severity, timestamp, msg, progname)
    "#{Kernel.caller[2]}: #{severity.upcase}: #{progname.gsub(/\n/, '').lstrip}\n"
  end
end

This little snippet is especially handy if you’re using an IDE like IntelliJ that’s smart enough to hyperlink the file path. Got an error being logged in users_controller.rb on line 91? Click on the hyperlink and you’re there.

It works by accessing the execution stack Array returned by Kernel.caller. caller0 will refer to the line in Logger#add where format_message is called. caller1 is most likely going to point to one of the Logger#<severity> methods, one of [ debug, info, warn, error, fatal ]. This is the next level down in the stack, where the add method was called from. In most “application-level” code (code we actually write in our application, exterior to the benchmarking messages and such that Rails gives us for free), caller2 is going to be something in our application code itself, some place that we called logger.info, or logger.error, or whatever, from.

To get this installed in our environment we create a file development_logger.rb, containing the source above. Then, in development.rb (or the initializer block of environment.rb):

require 'development_logger.rb'

This will “monkey patch” the Rails Logger, effectively overriding it’s format_message method with our own mojo. Sure, it’s not perfect. But it works great for development purposes. A more elegant way to do the same thing would be to subclass Logger and then do something like:

config.logger = DevelopmentLogger.new(config.log_path)

However, this just doesn’t seem to have any effect when running script/server, which proceeds as if config.logger is set to the standard (unmodified) Rails Logger. I’m at a bit of a loss as to why. If anyone can explain why setting config.logger seems to have no effect, please (please!) let me know.

Episode 1: Caching with Instance Variables

Posted over 7 years back at Railscasts

Learn a quick way to improve performance. Just store the end result of an expensive command in an instance variable!

New Ruby/Rails IDE Comparison

Posted over 7 years back at zerosum dirt(nap) - Home

Oh, what an IDE snob I’ve become over the past couple years. I was an advocate of lightweight text editors for dev work for a long time, and was only truly bitten by the “heavyweight” environment bug once I was introduced to IntelliJ, which I can’t say enough nice things about. I was obviously pretty psyched when I found out that the Jetbrains team has decided to put some serious effort into a Ruby plugin. It’s become very usable in a very short amount of time, and I’m very happy with it, but it’s still relatively new and lacking some features.

This new comparison of Ruby / Rails IDEs does a great job summarizing the features in the three leading “heavyweight” IDEs for Ruby / Rails. Definitely check it out if you’re in the market for a good IDE. Interestingly, it seems like NetBeans is really giving both IntelliJ and RadRails a real run for their money, being the first of The Big Three to get a reasonable level of code completion working, amongst a plethora of other features.

Although I’ll probably download it and take a test drive, I’m pretty committed to IntelliJ at this point, so I’ll probably stick it out. But hopefully a little competition will keep things moving along at a good clip :-).

UPDATE: Switched over to NetBeans months ago and loving it! Nice work guys.

attr_readonly

Posted over 7 years back at zerosum dirt(nap) - Home

I was really shocked yesterday to discover that there’s literally no baked-in way to declare an ActiveRecord attribute as private or protected. Sometimes I don’t want publicly-accessible ActiveRecord attribute methods. Sorry, but not every field in my database should have a corresponding public mutator.

Consider an IP address field on a User model, that’s used to record the last IP the user logged in from. This is something that, although clients outside of the model should be able to access the value, they certainly shouldn’t be able to set it. Only the model itself should be able to update that field in the database. It’s private!

As another example, consider the case where a database field is just a cache of calculated relationship data. That cached value should never be set directly via a public instance method. One example is an average_rating as calculated from a bunch of user ratings on an article. Why would you want to expose a mechanism to set this directly? It should only be set through some sort of recalculate_average_rating public method that lives on your article model and fires as an after_save filter on a new rating object.

It turns out that counter_cache itself suffers from this same problem. Ticket #6896 in the Rails Trac points out this exact issue and proposes a solution in the form of attr_readonly. This is a solid solution, and would go a long ways towards enforcing proper encapsulation in AR.

I ranted about this yesterday in freenode and got yelled at a little bit. Yes, I was probably being a bit obnoxious, but I also don’t want the importance of this to be overlooked. The bottom line is that, like it or not, people are largely stupid. Programmers are not excluded from this. Even Ruby programmers, who admittedly are more self-conscious than most. If you give someone the opportunity to do something stupid, be it at the user level in your app, within your API, or right within the rest of your code base, they’ll do it. If we’re writing for an audience > 1, we should be writing code like we design user interfaces — the public methods available on our classes shouldn’t include things that give our audience permission to twiddle with the guts of that thing. This is a cornerstone of encapsulation and good OO design.

So how do you feel about all this? Is it not A Big Deal to you? To me, it’s a significant limiting factor when it comes to organizing large code bases amongst multiple programmers. If you’re the only guy working on a hobby app, then fine, I guess you can police yourself. Go somewhere and do your thing. But if you’re part of a larger team, or hope to build the foundation for something that can grow into a real enterprise-class app, then proper encapsulation is mandatory.

I really hope that patch gets committed, or that someone puts together a plugin that delivers the same functionality. In the meantime, sure, there are some things you can do that are better than nothing at all. Try declaring a public mutator in your class to override the AR attribute method of the same name and raise NoMethodError, or write a before_save filter that loads up the model into another variable and replaces the current field with it’s previous value. But just describing those “solutions” makes me feel somewhat ill. Rails is such a clean, elegant platform, and it needs a clean, elegant way to provide non-public methods for AR attributes. Let’s get that patch committed :-).

Breaking into Orkut with Mechanize

Posted over 7 years back at schadenfreude

Alternate title: Using Mechanize to Scrape Orkut for Love Letters from Lonely Brazilian Beauties and Translating their Wantings to English

Shout outs to Natai for the request.

In this tutorial we will use mechanize to login to Orkut, pull our scrap messages, we will assume the message is written in Portuguese and translate the message to English with Google Translate.

FUN!

Breaking into Orkut with Mechanize

Posted over 7 years back at schadenfreude

Alternate title: Using Mechanize to Scrape Orkut for Love Letters from Lonely Brazilian Beauties and Translating their Wantings to English

Shout outs to Natai for the request.

In this tutorial we will use mechanize to login to Orkut, pull our scrap messages, we will assume the message is written in Portuguese and translate the message to English with Google Translate.

FUN!

Breaking into Orkut with Mechanize

Posted over 7 years back at schadenfreude

Alternate title: Using Mechanize to Scrape Orkut for Love Letters from Lonely Brazilian Beauties and Translating their Wantings to English

Shout outs to Natai for the request.

In this tutorial we will use mechanize to login to Orkut, pull our scrap messages, we will assume the message is written in Portuguese and translate the message to English with Google Translate.

FUN!

Axiom Of The Empty Set

Posted over 7 years back at zerosum dirt(nap) - Home

A list without any items in it is still a list, I must insist. Just like a box without anything in it is still a box, or a Smurf village without any Smurfs (Smurves?) is still a frickin Smurf villiage..

But not according to the W3C, apparently. File this one under Rant Of The Day: none of the standard XHTML doctypes acknowledge the existance of the empty list.

<!ELEMENT ul (li)+>

That is, neither the <ul></ul> or the <ul/> representations are valid XHTML. This seems pretty broken to me at first glance, although I’m certianly willing to hear a rebuttal if anyone has one. Even if you don’t think it’s busted, it certainly adds complication to a very common web dev scenario…

So we have a list of resources in our view and there’s a good possibility that our list is empty. We want to be able to display that list, and have a button on the page that lets me add new items to the list with a little Ajax love so that we don’t have to reload the page. If we can represent an empty list in the page, it’s simple: we can just render :update in Rails…

render :update do |page|
  page.insert_html(:bottom, :my_list, "<li>#{item}</li>")
end

The code above assumes that there’s a DOM ID ‘my\_list’ that has 0 or more elements. But since XHTML won’t let us represent a 0-element list, the obvious thing to do is to bake some extra smarts into the update code.

Now it’s the job of the update to determine if a list exists, and if not to create it, but only for the special one-off case of the zero element list? Yuck, bleh. No thanks. This is probably what most people do, but it just strikes me as putting too much intelligence into something that’s supposed to be pretty dumb.

The alternate solution listed here also qualifies as a hack, no bones about it. But, at least in my opinion, it’s a somewhat more elegant hack than what was described above. File under simple Workaround Of The Day:

<ul id="list_things">
  <li class="invisible" style="display: none"/>
</ul>

Note that the W3C does support empty list item elements :-). Thanks to azta for the suggestion.