How to understand what those barbarians are doing over there and why your going to keep on hearing about it.
Amy Hoy talks about learning Rails, website usability and her upcoming book.
Obie and Matt reflect on the whirlwind of long beards, huge slides, and curly brackets known as RubyConf.
Thomas Fuchs talks about writing the script.aculo.us libraries and changing the world with Ruby.
Core developer Jamis Buck talks about Switchtower and working at 37signals.
Core developer and Typo blog creator Tobias Luettke tells about the history of the blog and how he designs beautiful software with Rails.
Random #ruby-lang snippet number 10357:
[6:58pm] <tsume> I wish I had ESP [6:59pm] <danfg> time travel beats esp anytime [7:00pm] <rtomayko> forward time travel or backward time travel? [7:01pm] <danfg> backward is better i think [7:01pm] <rtomayko> i concur
The axioms of web architecture and an invitation for big vendors to understand them.
Author Dave Thomas talks about the upcoming Rails book and how he started with Ruby.
Rails creator David Heinemeier Hansson talks about writing Rails.
For a really long time now I've wanted to get my development related projects, essays, and other ramblings into a more coherent place on the web. At the same time, I've been thinking a lot about this "movement" (for lack of a better word) that I've grown a small part of that is simplifying, humanizing, and opening up IT.
There's a significant portion of the development community that's just plain pissed off at the status quo and have begun pursuing radically different ways of building and delivering systems from what is now commonly accepted by the established industry. To our delight, we're finding that There's Plenty of Room at the Bottom and that the next big thing in IT should really be making it smaller.
We're also finding that this new-think will not be accepted by the mainstream analyst firms, tech publications, and vendor powerhouses on simple merit. Pimping the latest acronym from the latest vendor with the latest money is a much easier way of bringing in stupid amounts of cash than is trying to move an industry forward in providing real value to actual people. So screw you guys - your goals are in direct competition with those of my customers.
As you can see, I'm a little bitter... But I'm also excited - we're assembling a stack of tools and techniques that are demonstrably superior and yet somehow much simpler than those of the mainstream past. We're starting companies and controlling our future and it just feels right.
So I'm proud to announce that lesscode.org is open for business. It's still a work in progress but here are my goals for the place:
Advocate, communicate, and discuss the happenings of the
Provide tools and resources for like minded individuals to collaborate on projects (subversion, mailman, trac, etc).
I'm also prepared to chase funding in the form of hosting, bandwidth, admin time, etc. should their be significant interest in the concept.
This site will stick around but most of my development related posts will move to lesscode.org.
Warren points to one of the most concise and powerful recitations of GNU/Free Software philosophy I've ever seen. This message to the fedora-devel mailing list is in response to one of the many "we must work with the proprietary driver vendors" pundits:
Choosing slavery is not a freedom, it's waiving freedom.
Likewise, accepting non-Free(dom) terms is not freedom of choice, but subjugation to somebody else's power over you.
In the case of 3d binary drivers, they're exploiting your craving for 3D, and providing it at [the] cost of your loss of freedom.
It is your choice, in fact, but not much different than the choice a chemically (3D) addicted person has over buying the chemical [from] the local dealer. You may get the appeasing visions, but in the end you'll eventually suffer, and you seem to like enticing others to do it, which is reprehensible.
That's the closest thing I've seen to a solid line in determining whether you subscribe to Free Software philosophy - you either agree or you don't.
Me? I hope to be completely clean someday... (as I write this on my proprietary Macintosh listening to DRM protected songs on iTunes - crack!)
It turns out that my site has been down for almost a week now. I thought I had everything back up on Tuesday after moving from Columbus to Cleveland over the weekend. Today I found out that Adelphia blocks port 80 and so I had to move everything from running off the trusty P300 in my basement to my co-located box. I should have done that a long time ago anyway.
But yea, so I've been churning out mad content this week thinking everything was fine and after receiving zero feedback through the week, I started thinking maybe the stuff just wasn’t that interesting. But it’s absolutely interesting and now that it’s available I want you to read it :)
IBM poop heads say LAMP users need to “grow up” is an unfair attack on IBM (and Sabbah in particular) for all that is wrong in the enterprise IT space. It’s long, unstructured, and predictably politically incorrect. Oh, and it’s also true.
Cha-cha-cha-cha-changes… is a bit of rambling on the great job I just left and some extremely important things I learned there, which I offer to you completely free of charge.
Let’s do it:
According to Daniel Sabbah, general manager of IBM’s Rational division, LAMP — the popular Web development stack — works well for basic applications but lacks the ability to scale.
Nope. We call bullshit. After wasting years of our lives trying to implement physical three tier architectures that “scale” and failing miserably time after time, we’re going with something that actually works.
If you look at the history of LAMP development, they’re really primative tools … the so-called good enough model. The type of businesses being created around those particular business models are essentially going to have to grow up at some point.
No. The LAMP stack is a properly constructed piece of software. Features are added when an actual person has an actual need that arises in the actual field, not when some group of highly qualified architecture astronauts and marketing splash-seekers get together to compete for who can come up with the most grown-up piece of useless new crap to throw in the product.
The LAMP model works because it was built to work for and by people building real stuff. The big vendor / big tools model failed because it was built to work for Gartner, Forrester, and Upper Management whose idea of “work” turned out to be completely wrong.
Now you’re saying that the primitive yet successful LAMP model should adopt the traits of the sophisticated yet failing big vendor model.
I believe that in the same way that some of those simple solutions are good enough to start with, eventually, they are going to have to come up against scalability,Sabbah said during a press conference at the IBM Rational User Conference in Las Vegas.
Alright, that last jab may have been a bit unfair. I think what Sabbah
is really talking about is PHP. I can’t be sure but none of
Amazon, Ebay, or Google seem to be using PHP widely on their public
sites. But then again, they aren’t using Websphere/J2EE, .NET, or other
scalable physical three tier architectures either.
UPDATE: See comments for interesting notes on PHP usage at Yahoo!.
While we’re talking about architectures, I'd like to jump into a brief commentary on what’s really at the root of the debate here.
are two widely accepted but competing
general web systems architectures: the Physical Three Tier
Architecture and the Logical Three Tier Architecture. IBM (and all the
other big tool vendors) have been championing one of them and LAMP is a
good framework for the other (although you’ll rarely hear anyone admit
that LAMP provides an overall architecture).
The Physical Three Tier Architecture
Many large enterprise web applications tried really hard to implement a Physical Three Tier Architecture, or they did in the beginning. The idea is that you have a physical presentation tier (usually JSP, ASP, or some other *SP) that talks to a physical app tier via some form of remote method invocation (usually EJB/RMI, CORBA, DCOM) that talks to a physical database tier (usually Oracle, DB2, MS-SQL Server). The proposed benefits of this approach is that you can scale out (i.e. add more boxes) to any of the physical tiers as needed.
Great, right? Well, no. It turns out this is a horrible, horrible, horrible way of building large applications and no one has ever actually implemented it successful. If anyone has implemented it successfully, they immediately shat their pants when they realized how much surface area and moving parts they would then be keeping an eye on.
The main problem with this architecture is the physical app box in the
middle. We call it
the remote object circle of hell. This is
where the tool vendors solve all kinds of interesting
type problems using extremely sophisticated techniques, which introduce
one thousand actual real world problems, which the tool vendors happily
solve, which introduces one thousand more real problems, ad
It’s hard to develop, deploy, test, maintain, evolve; it eats souls, kills kittens, and hates freedom and democracy.
Over the past two years, every enterprise developer on the planet has
been scurrying to move away from this architecture. This can be
witnessed most clearly in the Java community by observing the absolute
failure of EJB and the rise of
lightweight frameworks like
Hibernate, Spring, Webwork, Struts, etc. This has been a bottom up
movement by pissed off developers in retaliation to the crap that was
pushed on them by the sophisticated tool vendors in the early century.
Which brings us nicely to an architecture that actually works some times and loves freedom.
The Logical Three Tier Architecture
More specifically, the
Shared Nothing variant of the Logical
Three Tier Architecture says that the simplest and best way to build
large web based systems that scale (and this includes enterprise
systems goddamit) is to first bring the presentation and app code
together into a single physical tier. This avoids remote object hell
because the presentation code and the business logic / domain code are
close to each other.
But the most important aspect of this approach is that you want to push all state down to the database. Each request that comes into the presentation + app tier results in loading state for a set of objects from the database, operating on them, pushing their state back down into the database (if needed), writing the response, and then getting the hell out of there (i.e. releasing all references to objects loaded for this request, leaving them for gc).
That’s the rule.
So the physical database tier and the physical presentation + app tier make up our logical three tier architecture but I'd like to talk about one other latch-on piece of this setup because it’s interesting to contrast it with how the Physical Three Tier purists deal with the same problem.
Fine Grained Caching
Some mechanism for caching becomes really important when you decide that you are spending too much money on hardware (note that both of these architectures will scale up and out, on each physical tier independently, for as far and wide as you can pay for hardware). Adding some form of caching reduces the amount of hardware needed dramatically because you've reduced utilization somewhere.
In the physical three tier architecture, there is generally a lot of sophisticated mechanisms for caching and sharing object state at a very granular level in the app tier to reduce utilization on the the database and increase response time. This is cool and all but it increases utilization on the app tier dramatically because so much time is now spent managing this new state.
The introduction of state (even just a little state for caching objects) forces the app tier to take on a lot of the traits of the database. You have to worry about object consistency and be fairly aware of transactions. When that’s not fast enough what ends up happening is that more fine grained caching is added at the presentation tier to reduce round trips with the app tier.
Now you have three places that are maintaining pretty much the same state and that means you have three manageability problems. But this is, you know, cool because it’s really complex and sophisticated and the whiteboard looks interesting and lots of arm waving now.
Screw Fine Grained Caching
Shared Nothing says,
screw that – the database is the only thing
managing fine grained state because that’s it’s job, and then throws
up caching HTTP proxy server(s) in a separate (and optional) top
physical tier. Cached state is maintained on a much simpler, coarse
grained level with relatively simple rules for invalidation and other
When the Shared Nothing cache hits, it provides unmatched performance because the response is ready to go immediately without having to consult the lower tiers at all. When it misses, it misses worse than the fine grained approach because chances are good you’ll be going all the way to the database and back. But it turns out that it usually doesn’t matter. My experience says that you get as good or better performance with the coarse grained approach as you do with the fine grained approach for much less cost, although it’s hard to measure because the savings are distributed in very different ways.
The Shared Nothing + Caching Proxy setup scales like mad and I don’t just mean that it scales to really massive user populations. It scales low too. It’s easy to work with when you’re developing and testing on a single machine. It’s easy to have a simple prototype app done in a day. It’s easy to teach someone enough that they can go play and figure stuff out as they go. It’s easy to write tests because the entire system is bounded by the request and there’s no weird magic going on in the background.
The big vendor / big tool architectures sacrificed simplicity and the ability to scale low because they decided that every application was going to have one million users and require five 9’s from the first day of development.
As I write this, Bill de hÓra postulates: All successful large systems were successful small systems. I believe him and what that means to us right now in this article is that it is exceedingly hard to build large systems with the big vendor / big tool approach because it is exceedingly hard to build small systems with the same.
Let’s get back to the woodshed
While Sabbah was critical of LAMP’s capabilities, he said IBM is going to ensure companies which started with that model will be able to “grow and change with the rest of the world”.
He believes most businesses want technology that is stable, evolutionary, historical and had support.
L A M P = (S)table (E)volutionary (H)istorical (S)upport
“What we are trying to do is make sure businesses who start there [with LAMP] have a model, to not only start there but evolve into more complex situations in order to actually be able to grow,” he said.
This is where I really wanted to jump in because I think this mentality is holding back adoption of very simple yet extremely capable technology based purely on poor reasoning. This view of systems design says that complexity is required if growth is desirable and that complex situations can only be the result of complex systems.
There’s a guy who just spent 50 years or something locked in a room writing a 1200 page book proving that this is just wrong. It would appears that there is very little relationship between the complexity of a program and the complexity of the situation it produces.
The complexity for complexity mindset is the bane of a few potentially great technologies right now:
- Static vs. Dynamic Languages
- J2EE vs. LAMP
- WS-* vs HTTP
I like to complain when someone calls Python a
because the connotation is that it is simple. But it is simple, right?
So there shouldn’t be any complaining. I'm not objecting to someone
calling Python simple, I'm objecting to then saying that because it is
simple, it must only be capable of simple things.
The Need For Complex Systems
“You've seen us do a lot with PHP and Zend and you’ll see us do more. I can’t say more. It [PHP] needs to integrate with enterprise assets but it needs to remember where it came from and where its value was. It can’t start getting too complex as it will lose its audience,” Sabbah said.
The need for complex systems in the enterprise was and still is greatly overestimated. The trick isn’t to make PHP more complex, it’s to make the enterprise less complex. You need to equate complex requirements with complex systems less and start asking “do we really need this?” more.
The funny thing about all this is that my opinion on this matter has formed largely based on concepts that you guys told me, so I'm sure you’ll pull through on this one.
The site is back up after two days of downtime but if my theory is correct then no one noticed. Our records indicate that 95% of the (20) people reading this site do it via a proper RSS reader, so no one should have noticed a difference between the site being down and me just going on a quiet spell for a few days. The other traffic comes from search, and seeing as how my pages look almost flawless coming from google’s cache, I don’t see why I have to care about downtime all that much. Which is good because this site runs on and old P300 GNU/Linux box off my cable connection and has been since 2003 (talk about the ‘ilities!).
Anyway, cha-changes… I usually don’t get very personal in posts — tending more toward commentary on larger issues — but there’s so much going on right now that I have to write some of this down if just for the archives.
One of the things I never talk about here is my day job. This isn’t because it isn’t related (most of my posts on technology are at least relevant to my day job) but because my employer had no formal policy on public discourse. That combined with the fact that I really enjoy being able to say whatever the fuck I want here led me to keeping work completely out of the picture.
But today, while waiting on The Exit Interview With The Human Resources Department, I was reading over my non-disclosure agreement when I noticed that it was fairly sane, forbidding only the disclosure of confidential company information. Me being employed there isn’t confidential so I guess it’s okay to say that I was employeed by Sterling Commerce from March 2000 until about five hours ago.
Sterling has been around a loooong time for a tech company. This PR blurb does a surprisingly good job of providing some actual information on the company’s background:
A pioneer of electronic data interchange (EDI) and secure file transfer technology, it has provided business process automation solutions to Fortune 500 companies and the world’s largest banks for 30 years. Formerly a division of Sterling Software, Sterling Commerce became an independent corporation in March 1996, through an initial public offering. The company quickly grew to become one of the world’s largest independent providers of multi-enterprise collaboration solutions. SBC Communications acquired Sterling Commerce in March 2000.
They’re one of the few NASDAQ tech companies that didn’t pop with the bubble. I attribute this mainly to the fact that they actually did something: boring old valuable-as-hell EDI and other tough business integration tasks. Most of their current initiatives involve doing the same boring old valuable-as-hell EDI stuff but this time over internet pipes instead of bisync modems.
I won’t say much more about Sterling because talking about it is still gray to me and I don’t want to piss anyone off (at Sterling). I really don’t even know if I can talk about talking about it.
I would like to mention one product I worked on there because it was a
kind of epiphany to me. The product was Webforms (rebranded
Web Forms it seems
<sigh>). It’s just a web app that let
smaller Mom-and-Pop shops talk EDI to all the big boys running them
thar' expensive mainframes and bisync modems.
Now this was an important situation for a couple of reasons. First,
Webforms was an extremely functional web app for the time (~2000). The
web was all about animated GIFs, Java applets,
tags, and was just very
oh-we-can-make-money-by-just-looking-really-cool-ish in general. And
these guys show me this plain old ugly white pages with big forms all
over the place web-app. Maybe you'd see a small company logo here and
there, maybe not. Fuck it, your company logo isn’t helping anyone with
It was the plainest, most vanilla piece of pure gold I'd ever seen. Clay Shirky figured out the internet in 1996, I figured it out in 2000 while at Sterling Commerce. The internet is here to do the exact same shit we did before the internet was here but cheaper, broader, and with less fuss. ROCK ON! Let’s do it. I'm pumped.
Unfortunately that leads me to the second reason Webforms was an enlightening experience. It was developed in a company that really hated the web. We owned a massive EDI VAN and we had desktop Windows and UNIX products… that let you talk to the VAN. This is what 2000 people at Sterling knew and understood. Combine that with the following snippet from whatis.com’s definition of VAN:
… Before the arrival of the World Wide Web, some companies hired value-added networks to move data from their company to other companies. With the arrival of the World Wide Web, many companies found it more cost-efficient to move their data over the Internet instead of paying the minimum monthly fees and per-character charges found in typical VAN contracts.
And then these Webforms guys go and plop a web app down in the middle of all this – it wasn’t pretty. Needless to say, excitement about the product never really made it out of development for the years I worked on it but seems to finally be picking up a little now.
So the other big thing “I got” about the internet was that providing real value would usually require being extremely disruptive, which would require pissing a lot of people off, which I will now dub Tomayko’s Law of The Internet:
To provide value on the internet, you must piss someone off.
- Napster / RIAA
- Wikipedia / Britannica
- RSS / Journalism
- Skype / The Entire Phone Industry (holy shit!)
- Free and Open Source Software / Microsoft (the whole industry is being flipped on its head in case no one noticed)
- BitTorrent / Television (yes, bittorrent will replace your
television stationswhether you like it or not).
- Google / Yahoo
- Corey Doctorow / Disney
- And the list goes on…
All provide value by pissing someone off.
Notable exceptions from this list are AOL, WS-*, Semantic Web, Java Applets, RealAudio streaming, Microsoft Passport, top posting, and DRM. Each of these technologies piss everyone off; this somehow lessens the value dramatically. So the trick is to find a group that’s roughly the size of, let’s say, an industry and piss them off as much as possible.
That’s an insanely brief description of what I've learned at Sterling and this one is already running long so more on the ch-changes later.