Ruby Science: Extracting and Using Classes

Today’s release of Ruby Science includes three new chapters. If you’re already reading Ruby Science, make sure to grab the latest version.

This week’s updates include tips on safely extracting classes, as well as a specific example of extracting Value Objects. In addition, we discuss how you can use Ruby’s classes-as-objects attitude to eliminate the need for most abstract factory classes.

The book is a work in progress, and currently contains around 123 pages of content. A $49 purchase gets you access to the current release of the book, all future updates, and the companion example application. In addition, purchasers have the ability to send thoughtbot their toughest Ruby, Rails, and refactoring questions.

Get your copy of Ruby Science today.

Pick your compass metric

One of the most important tasks as a leader in a startup is to pick the right metric to track. This is often referred to as the 'compass metric' because it will be your compass for growth. It's important to note that 'compass metrics' will likely change over the lifetime of a business.

Let’s say you started a new social network. When it’s time to pick your first compass metric, you settle on tracking the sum of people who log in on a given day: Daily Active Users ('DAU').

Good pick. It’s hard to misrepresent a metric like DAU. If people don’t like your service, they will likely not return, and your DAU will fall over time. Another metric you could have picked is daily sign ups. This would also give you a good sense of progress, but it’s not nearly as robust. Although you want sign ups, they aren’t necessarily synonymous with the health of your business. By spending a lot of money on marketing you might get a lift in sign ups, but if none of these sign ups ever return, your metrics would tell a misleading story. Daily sign ups look great, but without daily active (and engaged) users you’re just filling up a leaking funnel.

The metric you decide on will be your compass for growth. You implicitly tell your team that if someone moves this metric in the right direction they are doing a good job. If you launch a lot of experiments you can sometimes allow people to use other metrics for some early readings, but you should only conclude the efficacy of your experiments by checking the impact on your compass metric.

After you settled on the compass metric, you have to choose a time frame. This is important because you want to have a very tight feedback loop. If you have a 6-month sales cycle and your chosen metric is bookings, then it’s very hard to properly experiment.

Lastly, you have to design the user interface for your tracking. How exactly are you going to keep your entire team informed? From which angle are you going to look at your metric? As an absolute number ($10,000 in bookings this month), or as a relative number (30% growth over the last quarter)?

Let’s take Shopify as an example. We are a fairly complex business with (thankfully) many different streams of revenue. Most of our money comes from customers that pay us subscription fees. After a lot of experimentation, we concluded that the best compass metric for Shopify is weekly CMRR growth.

CMRR?

CMRR stands for Committed Monthly Recurring Revenue. From Bessmer’s cloud computing law #2:

[…] To achieve better business visibility, most top performing cloud companies focus on Annual Contract Value (ACV) or Monthly Recurring Revenue (MRR) - the combined value of all of the current recurring subscription revenue - instead of bookings. We recommend companies actually take this a step further and track the forward view of Committed Monthly Recurring Revenue (CMRR). The CMRR differs from the MRR in two ways. Firstly, it includes both “in production” recurring revenues of the customer and the signed contracts going into production. Secondly, it is reduced by “churn” which is the MRR expected to be lost from customers that are anticipated to be ending service in the future. CMRR gives you the most pure forward view of the “steady state” revenue of the business based on all the known information today. This is the single most important metric for a cloud business to monitor, as the change in CMRR provides the clearest visibility into the health of any cloud business.

In short, you take the monthly value of all your active customers, plus committed sign-ups and upgrades, minus the monthly value of all your downgrades and churn. Then you calculate the growth of that number compared to last week.

Our internal goal is to reach 3% weekly growth, a very ambitious number given our size. The user interface is simple. Monday mornings, our system sends an email to the team:

Red ✘ if we fell short, green ✔ if we made it. Everyone gets it.

Most groups that have a direct CMRR impact also get their own versions of this email that reports how their group did compared to everyone else.

Weekly meeting

Every Thursday we have a quick funnel meeting. This meeting is attended by everyone who has a direct impact on the CMRR number, including those in sales, marketing, partnerships, etc. Everyone in attendance shares two things:

• What have we learned this week
• What we are going to do differently next week

This is the motor of a fast-growing multi-million dollar venture-backed business. Our growth comes from a combination of choosing the right time intervals (1 week), the right compass metric (CMRR), and the right perspective (week over week growth). There is only a minor amount of support scaffolding built into this structure: one email and one meeting a week. Everything else flows from that.

thoughtbot Drinkup in Paris, France I’ll be in Paris next...

thoughtbot Drinkup in Paris, France

I’ll be in Paris next week and we’re sponsoring a social event at MKP Opera on Thursday, February 28th at 7:30pm. RSVP here

I hope to see you there.

I’ll also be holding office hours while in Paris, so if you won’t be able to make it, or your like to get together to discuss technology, design, or business tweet me at @cpytel and we’ll coordinate a time.

2.0.0.pre10

Sure enough, the bug fix in 2.0.0.pre9 had a regression: some of the cancel links were broken. So here’s a 2.0.0.pre10. I definitely hope to have 2.0.0 final available soon.

We’ve also added a new tutorial upgrading a Hobo 1.0 app to Hobo 2.0.

2.0.0.pre10

Sure enough, the bug fix in 2.0.0.pre9 had a regression: some of the cancel links were broken. So here’s a 2.0.0.pre10. I definitely hope to have 2.0.0 final available soon.

We’ve also added a new tutorial upgrading a Hobo 1.0 app to Hobo 2.0.

Merge vs. rebase

The short:

• git pull --rebase instead of git pull

• git rebase -i @{u} before git push

• (on “feature”) git merge master

• (on “master”) git merge -no-ff feature

  However if “feature” contains only 1 commit, avoid the merge commit:
  (on “master”) git cherry-pick feature

The long:

If you enjoy this post, check out my git tips you didn’t know about!

Avoid merge commits that result from git pull

When you want to push your changes to a branch, but someone else already pushed before you, you have to pull in their changes first. Normally, git does a merge commit in that situation.

Such merge commits can be numerous, especially between a team of people who push their changes often. Those merges convey no useful information to others, and litter the project’s history.

You should always pull with git pull --rebase. To avoid having to always specify that option for pulls from a remote, git can be instructed to configure new tracking branches with this behavior:

git config --global branch.autosetuprebase always

Interactively rebase local commits before pushing

Run this every time before pushing a set of commits:

git rebase -i @{u}

The “u” stands for “upstream”, and it resolves to the SHA that is the latest state of the branch on the remote. Putting it simply, this command rewrites only the local commits which you’re about the push.

This gives you a chance to perform basic housekeeping before sharing your changes, such as squashing related commits together and rewording commit messages if they’re too long or not descriptive enough.

Suppose you have a set of 4 commits (newest first):

[D] oops! fixed typo in feature A
[C] bugfix for change B
[B] another change
[A] yay new feature!

You definitely want to squash A+D and B+C together, while still keeping A and B separate. The philosophy behind this is: don’t make bugs or typos a part of your project’s history if you can help it.

Integrate changes from master into a feature branch with merge

If you’re working on a long-lived feature branch, it pays off to sometimes merge in the master branch (assuming “master” is the main development branch) to verify that they are compatible and to get the latest bug fixes.

You could also rebase the current branch on top of master, but the rebase has shortcomings:

• If multiple people work on the feature, rewriting history complicates their workflow.
• Merge conflicts can be easier to deal with during merge than the more numerous, smaller conflicts during rebase.

Record a merge commit when a feature lands into master

After working on a feature/topic branch, merge it in master like so:

git merge --no-ff feature

The --no-ff flag ensures there will always be a merge commit, even when technically not necessary. Merge commits are useful because they convey the following information:

• where do changes come from (in this case: the “feature” branch);
• when were the changes merged and by whom, possibly indicating a code review (if that’s part of your development process);
• keeps commits related to this feature/topic grouped together.

Sometimes a topic branch will consist only of a single commit, especially for bug fixes. When merging it in, I often decide not to record the merge commit for a single commit because that merge information is less useful to the team:

• where changes come from is irrelevant, because the branch was likely short-lived, so people in the team haven’t developed familiarity with it;
• when is already recorded in the commit itself as committer timestamp;
• there’s nothing to group together.

So I merge a single commit like so in master:

git cherry-pick feature

Way Beyond Weight

This 80-minute, incredibly well-produced documentary gives us a personal look into the catastrophic impact that junk food corporations and other sugary products have on Brazilian youth, where 33% of all children are overweight or obese.

The key problem is demonstrated by frequent interviews of both children and their parents: they are mostly unaware of just how much sugar is contained in innocently packaged snacks that big brands market to kids.

What happens when RSpec runs, or, what I think about testing with blocks

Welcome to part two of the the post series which will hopefully cauterize the bleeding stump that is my Ruby Testing Quest.

This time, we will take a not-too-deep dive into how RSpec works. Last time we looked at MiniTest; if you haven’t already read that, it might be a better place to start than this.

Let’s get going.

A simple RSpec example

Here’s a simple RSpec example.

require "rspec/autorun"

describe "an object" do
  before :all do
    @shared_thing = Object.new
  end

  before :each do
    @something = Object.new
  end

  it "should be an Object" do
    @something.should be_an(Object)
  end

  describe "compared to another object" do
    before :each do
      @other = Object.new
    end

    it "should not be equal" do
      @something.should_not == @other
    end
  end

  after do
    @something = nil
  end
end

This is obviously extremely dull and pointless – just like the minitest one – but it contains just enough to exercise the major parts of RSpec that I care about. It’s actually slightly more sophisticated than the example that I used for MiniTest, because RSpec provides a couple of notable features that MiniTest doesn’t provide. Specifically, these are before :all setup blocks, and nested groups of tests¹².

I’m not particularly interested in looking at the other obvious distinguishing features of RSpec, like matchers and the BDD-style “should” language, as these aren’t actually a part of the core RSpec implementation³.

The two hallmark attributes here that I am interested in are:

• grouping test definitions within blocks (as opposed to classes)
• defining test behaviour using blocks (as opposed to methods)

Running the test spec

The simplest way of running this spec would be to save as something_spec.rb and run it from the command-line.

$ ruby something_spec.rb
..

Finished in 0.00198 seconds
2 examples, 0 failures
[Finished in 0.5s]

So – what’s actually happening here?

Autorun

As with the minitest example, the first line loads a special file within the test library that not only loads the library, but also installs an at_exit hook for Ruby to run when the interpreter exists.

In RSpec’s case, this is defined in RSpec::Core::Runner.autorun. This calls RSpec::Core::Runner.run with ARGV and the stderr and stdout streams.

In contrast with MiniTest, RSpec parses the options at this point, and will try to determine whether or not to launch using DRb. In most cases it will create an instance of RSpec::Core::CommandLine with the parsed options, and then calls run on that instance.

Within the run method, some setup happens (mostly preamble to be output by the reporter, which is set via the configuration). Then we iterate through all of the “example groups”, returned by RSpec::world.example_groups⁴.

Let’s take a diversion to see how things actually get into RSpec::world.example_groups.

Your example groups

Consider our example spec again. At the top we have a call to describe:

describe "an object" do

The describe method is actually defined within the module RSpec::Code::DSL, but this module is extended into self at the top level of the running Ruby interpreter (which is main, a singleton instance of Object), making the methods in that module available to call in your spec files. You can actually see all of the modules that have been extended into this instance:

require "rspec/core"

self.class.ancestors
# => [Object, Kernel, BasicObject]

class << self
  ancestors
  # => [RSpec::Core::SharedExampleGroup, RSpec::Core::DSL, Object, Kernel, BasicObject]
end

# also, self.singleton_class.ancestors in Ruby 1.9

From this we can tell that the ancestors of Object are still just Kernel and BasicObject, but the ancestors of the specific instance main includes a few extra modules from RSpec. Anyway, moving on…

describe and RSpec::Core::ExampleGroup

The describe method in RSpec::Core::DSL passes its arguments straight through to RSpec::Core::ExampleGroup.describe. This is where things get a little interesting. Within this inner describe method, a subclass of RSpec::Code::ExampleGroup is created, and given a generated name.

describe "a thing" do
  # your tests, um, I mean specs
end
RSpec::Core::ExampleGroup.constants
# => [:Nested_1, :Extensions, :Pretty, :BuiltIn, :DSL, :OperatorMatcher, :Configuration]

The class that was created is there: Nested_1. For each describe at the top level, you’ll have a new generated class:

describe "a thing" do
  # your specs
end
describe "another thing" do
  # more specs
end
RSpec::Core::ExampleGroup.constants
# => [:Nested_1, :Nested_2, :Extensions, :Pretty, :BuiltIn, :DSL, :OperatorMatcher, :Configuration]

After each subclass is created, it is “set up” via the set_it_up method, which roughly speaking adds a set of metadata about the group (such as which file and line it was defined upon, and perhaps some information about the class if it was called in the form describe SomeClass do ...), and stashes that within the created subclass.

module_eval

More importantly, however, the block which was passed to describe is evaluated against this new subclass using module_eval.

The effect of using module_eval against a class is that the contents of the passed block are evaluated essentially as if they were within the definition of that class itself:

class Lionel; end

Lionel.module_eval do
  def hello?
    "is it me you're looking for?"
  end
end

Lionel.new.hello?
# => "is it me you're looking for?"

You can see above that the behaviour is effectively the same as if we’d defined the hello? method within the Lionel class without any “metaprogramming magic”⁵.

It’s because of module_eval that you can define methods within example groups:

describe "a thing" do
  def invert_phase_polarity
    # waggle the flux capacitor or something
  end
end

RSpec::Core::ExampleGroup::Nested_1.instance_methods(false) # false means don't include methods from ancestors
# => [:invert_phase_polarity]

These methods are then effectively defined as part of the Nested_1 class that we are implicitly creating. This means that methods defined in this way can be called from within your specs:

describe "a method in an example group" do
  def the_method_in_question
    :result
  end

  it "can be called from within a spec" do
    the_method_in_question.should == :result
  end
end

We’ll see how this actually works a bit later. Knowing that the contents of the describe block are effectively evaluated within a class definition also explains what’s happening when the before methods are called:

  before :all do
    @shared_thing = Object.new
  end

  before :each do
    @something = Object.new
  end

Because this is evaluated as if it was written in a class definition, then before must be a method available on the ExampleGroup class. And indeed it is – RSpec::Code::ExampleGroup.before.

Well, almost.

Hooks

The before method actually comes from the module RSpec::Core::Hooks, which is extended into ExampleGroup. RSpec has a very complicated behind-the-scenes hook registry, which for the purposes of brevity I’m not going to inspect here..

The before method registers its block within that registry, to be retrieved later when the specs actually run.

Because I’m not going to really look too deeply at hooks, the call to the after method works in pretty much the same way. Here it is though, just because:

  after do
    @something = nil
  end

The spec itself

The next method that’s module_eval‘d within our ExampleGroup subclass is the it:

  it "should be an Object" do
    @something.should be_an(Object)
  end

Users of RSpec will know that you can call a number of methods to define a single spec: it, specify example, and others with additional meaning like pending or focus. These methods are actually all generated while RSpec is being loaded, by calls to define_example_method within the class definition of ExampleGroup. For simplicity’s sake (pending and focussed specs are somewhat outwith the remit of this exploration), we’ll only look at the simplest case.

When it is called, more metadata is assembled about the spec (again, including the line and file), and then both this metadata and the block are passed to RSpec::Core::Example.new, which stashes them for later.

Nesting

Within our outer example group, we’ve nested another group:

  describe "compared to another object" do
    before :each do
      @other = Object.new
    end

    it "should not be equal" do
      @something.should_not == @other
    end
  end

Just as the top-level call to describe invokes a class method on RSpec::Core::ExampleGroup, this call will be invoked against the subclass of ExampleGroup (i.e. Nested_1) that our outer group defined. Accordingly, each call to describe defines a new subclass⁶, stored as a constant within the top-level class: Nested_1::Nested_1. This subclass is stored within an array of children in the outer Nested_1 class.

Within the definition, our before and it calls evaluate as before.

Your spec, as objects

So, for every describe, a new subclass of ExampleGroup is created, with calls to before and after registering hooks within that subclass, and then each it call defines a new instance of RSpec::Core::Example, and these are stored in an array called examples within that subclass.

We can even take a look at these now, for a simplified example:

group = describe "a thing" do
  it "should work" do
    (1 + 1).should_not equal(2)
  end
end

group
# => RSpec::Core::ExampleGroup::Nested_1

group.examples
# => [#<RSpec::Core::Example:0x007ff2523db048
#      @example_block=#<Proc:0x007ff2523db110@example_spec.rb:7>,
#      @options={},
#      @example_group_class=RSpec::Core::ExampleGroup::Nested_1,
#      @metadata={
#        :example_group=>{
#          :description_args=>["a thing"],
#          :caller=>["/Users/james/Code/experiments/rspec-investigation/.bundle/gems/ruby/1.9.1/gems/rspec-core-212.#   2/lib/rspec/core/example_group.rb:291:in `set_it_up'", "/Users/james/Code/experiments/rspec-ivestigation/.#   bundle/gems/ruby/1.9.1/gems/rspec-core-2.12.2/lib/rspec/core/example_group.rb:243:in `ubclass'", #   "/Users/james/Code/experiments/rspec-investigation/.bundle/gems/ruby/1.9.1/gems/rspec-core-2.12.#  2/lib/rspec/core/example_group.rb:230:in `describe'", "/Users/james/Code/experiments/rspec-investigation/.#  bundle/gems/ruby/1.9.1/gems/rspec-core-2.12.2/lib/rspec/core/dsl.rb:18:in `describe'", "example_spec.#   r:6:in `<main>'"]
#        },
#        :example_group_block=>#<Proc:0x007ff255c11430@example_spec.rb:6>,
#        :description_args=>["should work"],
#        :caller=>["/Users/james/Code/experiments/rspec-investigation/.bundle/gems/ruby/1.9.1/gems/rspec-core-2.1.#   2/lib/rspec/core/metadata.rb:181:in `for_example'", "/Users/james/Code/experiments/rspec-investigation/.#  bundle/gems/ruby/1.9.1/gems/rspec-core-2.12.2/lib/rspec/core/example.rb:81:in `initialize'", #   "Users/james/Code/experiments/rspec-investigation/.bundle/gems/ruby/1.9.1/gems/rspec-core-2.12.#   2lib/rspec/core/example_group.rb:67:in `new'", "/Users/james/Code/experiments/rspec-investigation/.#   bndle/gems/ruby/1.9.1/gems/rspec-core-2.12.2/lib/rspec/core/example_group.rb:67:in `it'", "example_spec.#   r:7:in `block in <main>'", "/Users/james/Code/experiments/rspec-investigation/.bundle/gems/ruby/1.9.#   1gems/rspec-core-2.12.2/lib/rspec/core/example_group.rb:244:in `module_eval'", #   "Users/james/Code/experiments/rspec-investigation/.bundle/gems/ruby/1.9.1/gems/rspec-core-2.12.#   2lib/rspec/core/example_group.rb:244:in `subclass'", "/Users/james/Code/experiments/rspec-investigation/.#   bndle/gems/ruby/1.9.1/gems/rspec-core-2.12.2/lib/rspec/core/example_group.rb:230:in `describe'", #   "Users/james/Code/experiments/rspec-investigation/.bundle/gems/ruby/1.9.1/gems/rspec-core-2.12.#   2lib/rspec/core/dsl.rb:18:in `describe'", "example_spec.rb:6:in `<main>'"]
#      },
#      @exception=nil,
#      @pending_declared_in_example=false>
#    ]

Where example groups are nested, further subclasses are created, and stored in an array of children within their respective parent groups.

Almost there!

Phew. The detour we took when looking at this aspect of minitest was much shorter, but now that we understand what happened when our actual spec definition was evaluated, we can return to RSpec running and see how it’s actually exercised.

As we saw above, the describe method returns the created subclass of RSpec::Core::ExampleGroup, and when that is returned back in RSpec::Code::DSL#describe, the register method is called on it. This calls world.register with that class as an argument, where world is returned by RSpec.world and is an instance of RSpec::Core::World, which acts as a kind of global object to contain example groups, configuration and that sort of thing.

Calling register on the World instance stashes our Nested_1 class in an example_groups array within that world.

Our diversion is complete! You deserve a break. Go fetch a cup of your preferred delicious beverage, you have earned it!

Back in RSpec

OK, pop your brain-stack back until we’re in RSpec::Core::Commandline#run again. Our reporter did its preamble stuff, and we were iterating through @world.example_groups, whose origin we now understand.

For each example group, the run method is called on that class, with the reporter instance passed as an argument.

This gets a bit intricate, so I’m going to step through the method definition itself (for version 2.12.2) to help anchor things.

def self.run(reporter)
  if RSpec.wants_to_quit
    RSpec.clear_remaining_example_groups if top_level?
    return
  end

RSpec has a “fail fast” mode, where any single example failure will cause the execution of specs to finish as quickly as possible. Here, RSpec is checking whether anything has triggered this.

  reporter.example_group_started(self)

Next, the reporter is notified that an example group is about to start. The reporter can use this information to print out the name of the group, for example.

  begin
    run_before_all_hooks(new)

The run of the examples is wrapped in a block so it can catch any exceptions and handle them gracefully as you might expect.

The before :all hooks

The call to run_before_all_hooks is very interesting though, and worth exploring. A new instance of the example group is created. It is then passed into this method, where any “before all” blocks are evaluated against that instance, and then the values of any instance variables are stashed.

Consider our original example:

  before :all do
    @shared_thing = Object.new
  end

Given this, we’ll stash the value of @shared_thing (and the fact that it was called @shared_thing) for later use.

It’s actually quite easy to inspect the instance variables of an object in Ruby; try calling instance_variables, instance_variable_get and instance_variable_set on some objects in an IRB session:

class Thing
  def initialize
    @value = Object.new
  end
end

class OtherThing
end

thing = Thing.new
thing.instance_variables # => [:@value]
ivar = thing.instance_variable_get(:@value) # => #<Object:0x007fe43a050e30>

other_thing = OtherThing.new
other_thing.instance_variables # => []

other_thing.instance_variable_set(:@transplanted_value, ivar)
other_thing.instance_variables # => [:@transplanted_value]
other_thing.instance_variable_get(:@transplanted_value) # => #<Object:0x007fe43a050e30>

As you can see above, we can poke around with the innards of objects to our heart’s content. Who needs encapsulation, eh?

Why did RSpec have to create an instance of the example group class, only to throw it away after the before :all blocks have been evaluated? Because RSpec needs to evaluate the block against an instance of the example group so that it has access to the same scope (e.g. can call the same methods) as any of the specs themselves.

Running the example

Now we’re finally ready to run the examples:

    result_for_this_group = run_examples(reporter)

To understand this, we need to look at the definition of run_examples:

def self.run_examples(reporter)
  filtered_examples.ordered.map do |example|
    next if RSpec.wants_to_quit
    instance = new
    set_ivars(instance, before_all_ivars)
    succeeded = example.run(instance, reporter)
    RSpec.wants_to_quit = true if fail_fast? && !succeeded
    succeeded
  end.all?
end

This method iterates over each Example that was stored in the examples array earlier, filtering them according to any command-line parameters (though we are ignoring that here). The most relevant part for us lies in the middle:

    instance = new
    set_ivars(instance, before_all_ivars)
    succeeded = example.run(instance, reporter)

A striking parallel with MiniTest

Another new instance of the ExampleGroup subclass is created. Remember, RSpec created one instance of the class for the before :all blocks, but now it’s creating a fresh instance for this specific spec to be evaluated against.

Thinking back to how MiniTest works, there’s a striking parallel: where MiniTest would instantiate a new instance of the MiniTest::Unit::TestCase for each test method, RSpec is creating a new instance of the ExampleGroup subclass to evaluate each Example block against.

Instances of this class are used so that any methods defined as part of the spec definition are implicitly available as methods to be called in the “setup” and “test” bodies (see the module_eval section above). Not so different after all, eh?

Next, the instance variables that we stashed after evaluating the before :all blocks are injected (effectively using instance_variable_set as we saw above) into this new instance, which will allow the spec to interact with any objects those blocks created. It also means that these values are shared between every spec, and so interactions within one spec that changed the state of one of these instance variables will be present when the next spec runs.

Finally, the #run method on the Example subclass is called, passing the ExampleGroup instance and the reporter. Down one level we go, into Example#run…

The spec finally runs

Here’s the full definition of RSpec::Core::Example#run:

def run(example_group_instance, reporter)
  @example_group_instance = example_group_instance
  @example_group_instance.example = self

  start(reporter)

  begin
    unless pending
      with_around_each_hooks do
        begin
          run_before_each
          @example_group_instance.instance_eval(&@example_block)
        rescue Pending::PendingDeclaredInExample => e
          @pending_declared_in_example = e.message
        rescue Exception => e
          set_exception(e)
        ensure
          run_after_each
        end
      end
    end
  rescue Exception => e
    set_exception(e)
  ensure
    @example_group_instance.instance_variables.each do |ivar|
      @example_group_instance.instance_variable_set(ivar, nil)
    end
    @example_group_instance = nil

    begin
      assign_generated_description
    rescue Exception => e
      set_exception(e, "while assigning the example description")
    end
  end

  finish(reporter)
end

For our purposes, we again only need to consider a small part. Once all the reporter and “around” block housekeeping has taken place, the essential core of the example is run:

          run_before_each
          @example_group_instance.instance_eval(&@example_block)
        rescue Pending::PendingDeclaredInExample => e
          @pending_declared_in_example = e.message
        rescue Exception => e
          set_exception(e)
        ensure
          run_after_each

The call to run_before_each introspects the hook registry and evaluates every relevant before hook against the ExampleGroup instance. In effect, this will find any before blocks registered in this example group, and then any blocks registered in any parent groups, and evaluate them all in order, so that each nested before block runs.

Then, the spec block (stored in @example_block) is evaluated against the ExampleGroup instance. This is where your assertions, or matchers, are finally – finally! – evaluated.

If there was a problem, such as a matcher failing[^matcher-exception] or an exception being raised, then the exception is stored against this Example for later reporting. Just as MiniTest assertions raise an exception when they fail, RSpec matchers raise an RSpec::Expectations::ExpectationNotMetError exception. It seems this is the universal way of halting execution when a test fails⁷. Another hidden similarity between RSpec and MiniTest!

As in MiniTest, whether or not the spec failed or an exception occured, an ensure section is used to guarantee that run_after_hooks is called, and any teardown is performed.

After the specs have run

Once all the specs in this example group have run, all the examples in any subclasses are run (recall that the inner describe stashed the nested ExampleGroup subclass in an array called children). We map each ExampleGroup subclass to the result of calling run on it, which starts this whole process again, for every nested example group. Whether or not this group passed or failed overall is then determined using simple boolean logic:

    results_for_descendants = children.ordered.map {|child| child.run(reporter)}.all?
    result_for_this_group && results_for_descendants

As we leave the call to ExampleGroup#run, we run any corresponding after :all blocks, and also clear out our stash of before :all instance variables, because they are no longer necessary.

  ensure
    run_after_all_hooks(new)
    before_all_ivars.clear
    reporter.example_group_finished(self)
  end
end

Finishing up

You can once again pop your brain-stack back until we’re in RSpec::Core::Commandline#run.

Having run all of the example groups, RSpec will do a little bit of tidy up, and finally return back up through the stack. Along the way printing the results of the run to the console is performed, before interpreter finally, properly quits.

Phew. You deserve another rest.

Testing with blocks

In contrast to the class-based implementation with MiniTest, we’ve now seen how a block-based test framework can work. In a nutshell, it can be characterised in a couple of key ways:

• the stashing of behaviour blocks, later evaluated using instance_eval against clean test-environment instances (see this section of the MiniTest article for what I mean by “test environment”);
• using module_eval and subclassing to ensure method definition matches programmer expectation.

I would say these two aspects are the hallmark attributes of an RSpec-style test framework. The other notable aspect is the ability to nest example groups, and the subsequent necessity to be able to gather the implicit chain of setup blocks and evaluate them against the test environment instance, but this could be considered another example of using instance_eval.

Supporting method definition in example groups

One thing I’ve found particularly interesting is that RSpec ends up generating classes and subclasses behind the scenes. I believe this is almost entirely a consequence of wanting to support the kind of “natural” method definition within group bodies (see the module_eval section again).

If any test framework chose to not support this, there’s almost certainly no reason to create classes that map to example groups at all, and the setup and test blocks could be evaluated against a bare instance of Object.

Supporting nesting and dynamic composition

It’s clear that RSpec has more “features” (e.g. nesting, before :all and so on) than MiniTest (ignoring the many extensions available for MiniTest, the most sophisticated of which end up significantly modifying or replacing the MiniTest::Unit.run behaviour). I’m deliberately ignoring features like matchers, or a built-in mocking framework, because what I’m most interested in here are the features that affect the structure of the tests.

It’s certainly possible to implement features like nesting using subclasses and explicit calls to super, but this is the kind of plumbing work that Ruby programmers are not accustomed to accepting. By separating creation of tests from Ruby’s class implementation, the implicit relationships between groups of tests can take this burden instead, and behaviours like before :all, which have no natural analogue in class-based testing, are possible.

Now, you may believe that nesting is fundamentally undesirable, and it is not my present intention to disabuse you of that opinion. It’s useful (I think) to understand the constraints we accept by our choice of framework, and I’ve certainly found my explorations of MiniTest and RSpec have helped clarify my own opinions about which approach is ultimately more aligned with my own preferences. While I wouldn’t say that I’m ready to jump wholesale into the RSpec ecosystem, I think it’s fair to say that my advocacy of class-style testing frameworks is at an end.

RSpec and Kintama

I started this exploration because I wanted to understand the relationship between the software I have accidentally produced and what’s already available. I already had strong suspicions that any block-based testing implementation would converge on a few common implementation decisions, and while I have now identified a few interesting (to me) ways in which RSpec and Kintama diverge, the essential approach is the same.

In the final article in this triptych (coming soon, I hope), I’ll walk through Kintama and point those out.

Episode 36: A gem called exploit

Episode 36: A gem called exploit:

This week Ben Orenstein is joined by Nick Quaranto, developer at 37signals and one of the maintainers of RubyGems.org. Nick and Ben discuss the just released Basecamp iOS app, the architecture of the app, the origins of the app and how it became what it is today, and RubyMotion in general. They then move on to discuss the recent RubyGems.org cracking, the mechanism behind it, the process of restoring the service, and how it might affect RubyGems going forward. They then circle back to talk more about RubyMotion, testing, working at 37signals, Co-work Buffalo, OpenHack, and good coffee.

• Episode Notes and Links
• Subscribe via iTunes
• Subscribe via RSS
• Listen via Stitcher
• Direct Download

Running a SaaS? Here are some services you’ll find useful

This December, our Time Tracking app Freckle will turn five. I’ve learned a lot about running a SaaS over these years, from making the right business decisions to choosing the technology to go with them.

The single most important thing you have to do as a business is make money. You need to pay your bills and put food on the table. At the same time you don’t want to compromise what you’ve set out for, in our case building the best time tracking tool there is and making it fun and awesome.

It follows that you need to have the most time available to actually talk to customers, refine the product and add awesome features—without losing the focus on what you’ve set out for.

In other words, you want to minimize time spend on things that other people probably know more about, including obvious things like taxes and accounting, but also keeping a SaaS up and running.

Here’s some tools we happily pay money for to keep some of the work of our backs:

• Travis Pro for continuous integration. We used to run our own CI server, no more. One more thing we don’t have to keep running
• NewRelic, which monitors not only the performance of our code and servers, but also external web services. This enables us to watch for performance issues with new releases as well as pro-actively act before something starts to slow down Freckle.
• Postmark sends our transactional email. No need for us to run our own mail server plus we know we can rely on deliverability. A must if your app sends mail.
• Honeybadger for error monitoring & tracking
• DocRaptor, a hosted version of PrinceXML, for converting HTML to PDFs. We literally implemented PDF downloads for invoices in one hour of coding with it (that includes writing tests and all).
• logentries.com keeps a tab on our logfiles, and provides instant search in case we encounter an issue. Much faster to quickly locate logs as you can view logs of the same time period across servers combined into one view.
• Dome9 provides a SaaS firewall solution so we don’t have to mess with iptables or other weird things manually. They have an iPhone app to on-demand open up ports we need to administrate our servers.
• Webmon and Pingdom allows us to monitor world-wide service availability and response time, as well as see if our DNS resolves globally. Pingdom also provides our status website without further configuration necessary (it took 5 minutes to set up).
• Dead Man’s Snitch emails you when periodical tasks (cronjobs) aren’t running.
• PagerDuty collects alerts from various services and gives us a central place from where to distribute alerts by text message and emails. This saves time by not having to configure all monitoring services separately.
• Tinfoil and Trustwave do security scans of our infrastructure, and tell us when it’s time to install security updates.
• KISSmetrics for metrics and event tracking and Customer.io for sending out on-boarding emails, as well as emails about features.

If there’s someone that can do it for you (probably better than you can) for a price that makes sense, then you’d be stupid to build it yourself (and by the way, this list is not exhaustive, there’s even more services we use).

There’s a thriving ecosystem of apps and services that help you running your SaaS—make good use of it!

Class-based generic views in Django

I have a confession to make: Even though thoughtbot is mostly known for the work we do with Ruby on Rails, I’m a huge Django fan. Someone left the keys to the blog lying around, so I thought I’d take it for a quick joy ride around the one of my favourite Django features: class-based generic views.

Firstly, just to help anyone who’s not au fait with Django terminology catch up, a view deals with a request and does whatever needs to be done to produce the correct response. If Rails is your thing, you can think of it as being roughly equivalent to a controller action.

Class-based views were introduced in Django 1.3, but despite being around for a couple of years they’re still not as widely used as they should be.

What came before

Before the class-based view there was the humble function-based view. It was a simple time. Views would take a request and return a response. Here’s an example for displaying a single blog post:

def blog_post_detail_view(request, *args, **kwargs):
    """Displays the details of a BlogPost"""
    blog_post = get_object_or_404(BlogPost, pk=kwargs.get("pk"))
    return TemplateResponse(request, "blog/blog_post.html", {
        "blog_post": blog_post,
    })

The mechanism is undoubtedly simple, but the resulting code isn’t. It’s very dense, with a single function loading the context data and building the response. This small example isn’t too bad, but imagine if we added comments, and author information, and a list of related posts. We’d quickly get to something very unwieldy.

Class-based views

It would be great if we could encapsulate all of this logic in a class so that it was less dense and easier to extend. Conveniently Django 1.3 helps us to do just that by providing a View class that we can extend. Here’s our blog post detail view, refactored into a class:

class BlogPostDetailView(View):
    """Displays the details of a BlogPost"""

    def get(self, request, *args, **kwargs):
        return TemplateResponse(request, self.get_template_name(),
                                self.get_context_data())

    def get_template_name(self):
        """Returns the name of the template we should render"""
        return "blog/blogpost_detail.html"

    def get_context_data(self):
        """Returns the data passed to the template"""
        return {
            "blogpost": self.get_object(),
        }

    def get_object(self):
        """Returns the BlogPost instance that the view displays"""
        return get_object_or_404(BlogPost, pk=self.kwargs.get("pk"))

The readability and extensibility have definitely improved, but the code could still be better. There are a lot of small decisions encoded in the class that needn’t be here. In writing this code I had to decide what to call the template, what to call the URL keyword argument that contains the model’s primary key, and what to call the context variable that is passed to template. None of those decisions were hard to make, but none of them really matter that much. What matters is that they remain consistent between different views, so that developers don’t have to waste time looking for things only to discover that this view doesn’t quite work like the last one they used.

In a nutshell, what this code needs is conventions. Application level conventions are good, but framework level conventions are better: The same developer can quickly understand many applications and easily move between projects or even companies without needing as much time to get up to speed. As consultants, when we’re working with new clients framework level conventions are invaluable. Rails is famed for its opinionated stance on convention over configuration, while Django is generally quieter on the subject. This might lead you to believe that Django doesn’t provide any conventions, but there’s more to Django than meets the eye.

Class-based generic views

Convention is where the “generic” part of “class-based generic views” comes into play. Django provides subclasses of View for a variety of common situations that are packed full of conventions and take all of those pesky little decisions out of our hands.

Using the generic DetailView, which displays details of a single model instance, we can boil the previous example down to a few simple lines:

class BlogPostDetailView(DetailView):
    """Displays the details of a BlogPost"""
    model = BlogPost

Convention and configuration

Of course, convention stops being helpful as soon as you want to do something unconventional. That’s where configuration comes into its own. Thankfully, Django’s class-based generic views provide both by using the Template Method pattern which makes it very easy to customise each part of the generic process.

Let’s do something less conventional and update our BlogPostDetailView to only display posts with a published flag set to True. We can do this by providing the view with a QuerySet to use as the basis of its query:

class BlogPostDetailView(DetailView):
    """Displays the details of a BlogPost"""
    model = BlogPost
    queryset = BlogPost.objects.filter(published=True)

Alternatively, we can go one step further and override the get_queryset method and use different querysets based on the properties of the request:

class BlogPostDetailView(DetailView):
    """Displays the details of a BlogPost"""
    model = BlogPost

    def get_queryset(self):
        if self.request.GET.get("show_drafts"):
            return BlogPost.objects.all()
        else:
            return BlogPost.objects.filter(published=True)

The code is still very concise and readable. Template Method has allowed us to override one part of the algorithm without reimplementing the whole thing. When we return to this class in the future there aren’t dozens of lines of boilerplate code to read through to find the significant parts.

Behind the scenes, the DetailView class provides an implementation of get, which in turn calls the get_object method. get_object is the template method, so if we wanted to buck all of the conventions we could override it. Since we’re only concerned with changing the queryset, we can ignore the template method itself and turn our attention to get_queryset, which is one of the primitive operations that get_object uses. In the first example we take advantage of the default implementation of get_queryset, which will return self.queryset if we’ve provided it. In the second example we override get_queryset entirely. In both cases we keep the rest of get_object’s algorithm, including useful features like 404 Not Found responses and support for looking up the model instance by primary key or using a slug.

Knowing what to override

The downside isn’t reading existing views, but writing new ones. The documentation for this feature has improved significantly since Django 1.3 was released, but there’s still something of a learning curve. Of course, you’re going to end up learning a set of conventions for your application, so it may as well be the one Django provides.

The good news is that these class-based generic views are very TDD friendly. If you are missing some required configuration attribute Django will raise an ImproperlyConfigured exception with a helpful message outlining your options (usually either setting an attribute, or overriding one or more methods that depend on the missing attribute). There’s rarely such a clear example of a failing test telling you exactly what to do next.

There are also some very helpful resources out there to get you started and to use as reference material when you’re up and running:

1. The official Django documentation on class-based views for a complete introduction than I’ve given here.
2. The official class-based views reference for specifics of individual classes and methods. In particular, this includes a “method flowchart” for each class which is helpful for figuring out what the Template Methods and their primitive operations are called.
3. The source code for the django.views.generic module is also a fun read.
4. Classy class-based views is an alternative set of documentation, which is particularly useful as a quick reference.

That’s all folks

I hope you enjoyed this look inside Django. Maybe we can do it again sometime?

2.0.0.pre9

I was really hoping to release Hobo 2.0.0 this week, but we did fix a couple of very minor bugs in the interim. It’s remotely possible that one of the bug fixes may impact your applications, so I’ve decided to release a Hobo 2.0.0.pre9 first. I expect 2.0.0 to follow in a couple of days.

Please test 2.0.0.pre9 against your application. It would also be helpful if you could try your application with config.hobo.dont_emit_deprecated_routes set to both true and false. This should not have any impact on your application except for differences in named routes. If there are other differences, please bring them to our attention; it indicates a probable bug.

I have also updated some of the marketing material. Please check it out and help make it better by clicking on those edit links!

Changes

• the overlap between accept_invitation and accept_invitation_from_email was confusing object_url.
• small change to DRYML to silence a warning about undefined attributes

2.0.0.pre9

I was really hoping to release Hobo 2.0.0 this week, but we did fix a couple of very minor bugs in the interim. It’s remotely possible that one of the bug fixes may impact your applications, so I’ve decided to release a Hobo 2.0.0.pre9 first. I expect 2.0.0 to follow in a couple of days.

Please test 2.0.0.pre9 against your application. It would also be helpful if you could try your application with config.hobo.dont_emit_deprecated_routes set to both true and false. This should not have any impact on your application except for differences in named routes. If there are other differences, please bring them to our attention; it indicates a probable bug.

I have also updated some of the marketing material. Please check it out and help make it better by clicking on those edit links!

Changes

• the overlap between accept_invitation and accept_invitation_from_email was confusing object_url.
• small change to DRYML to silence a warning about undefined attributes