Here are eight things my team has found to be true after working with Cucumber for about 6 months.

This post is for people who use cucumber. For a getting started tutorial, check out my earlier post on how to write a basic feature, the wiki, or the recent railscast.

1 – Don’t run everything at once

Many people use “rake features” after each change. This can be a waste.

Instead, as you work on a feature, you can run only certain features and scenarios:

1
2
3

rake features FEATURE=features/a_feature.feature

Even more granular, you can run specific scenarios with the feature, calling them by line number (from drnic).

1
2
3
4
5

cucumber features/a_feature.feature:23:70
# where 23 and 70 are lines beginning with "Scenario: ..."

2- Simplify common tasks

Typing “rake features FEATURE=name.feature:123” gets boring fast.

To combat this, here’s my shortcuts for your ~/.profile:

1
2
3
4
5
6
7
8
9
10
11
12
13

alias rf='rake features'

# For running specific features.
function rff {
 rake features FEATURE=features/"$1".feature
}

# For running specific line numbers
function rffl {
 cucumber features/"$1".feature:"$2"
}

3 – “That’s a huge amount of flexibility.”

How long is too long for a feature? How specific should your features be? How abstract?

David Chelimsky had a great reply to questions like this.

He basically said that the more detailed your features are, the more brittle they become and the less they can be reused across implementations / platforms. At the same time, the loss of specificity is a big deal for some people. To deal with this, he suggests having “a single scenario that describes the details,” plus “a lot more scenarios that are more declarative.”

The point is, do what works for you and your team.

For more on declarative vs imperative / detailed vs abstract, check out this post.

4 – Exploit the Cucumber formats

Cucumber can output results in many formats. You can use the normal dots, or show all the steps one by one, use HTML, or even use the ‘profiler’ format that will tell you about which steps are slowing down your tests the most.

To change the output style, change the following lines in lib/tasks/cucumber.rake

1
2
3
4
5

Cucumber::Rake::Task.new(:features) do |t|
  t.cucumber_opts = "--format progress"
end

The options are:

Progress – Dots for passing. Takes up the least lines of output. Useful for when you have enough steps that it becomes hard to find which one is failing b/c you have to scroll up so much.

Pretty – The plain text, with color.

Profile – A red/yellow format that gives the avg runtime of the longest steps.

Usage – Great for cleaning up step files. Gives you a list of all your steps, with the unused steps in red at the bottom.

5 – Speed is not your friend

Cucumber encourages full-stack testing. This is why we use it. But it’s easy to end up with a 6 minute test suite. So, before testing that case where a user has 20 cats, 40 dogs, and 15 carrots, consider whether you want to wait for that test to run 3000 times over the next months as you code.

Once you do end up with a 2-20 minute test suite, consider:

Seeding common data

Ben’s tip about seeding data helps immensely.

1
2
3
4
5
6
7
8
9

 
normal_cat = Cat.make
normal_hair = Hair.make

Given "I have the setup that we used to create 500 times indvidually" do
   @cat = normal_cat
   @hair = normal_hair
end

Testjour

Testjour is a gem for “Distributed test running with autodiscovery via Bonjour.” It helped one team take their test suite from 20 minutes to 2.

Deleting some tests (gasp!)

Chances are, you have some scenarios that can be trashed, or commented out. For example, if you have a huge suite of tests that run on a part of your app that is very decoupled and not going to change for several weeks or months, but adds 1 minute to your suite runtime, consider commenting out the non-essential components. You lose some coverage, but if you are doing BDD, then saving 1 minute worth of testing time may be worth it.

6 – Break up longer features

If you have a feature that has multiple parts, use whitespace in between groups punctuated by Thens.

1
2
3
4
5
6
7
8
9
10
11
12

Given foo
When x
Then y

When z
Then p

Given f
When t
Then q

This is easier to read.

7 – Avoid relying on coincidence

It’s easy to be cheap and rely on things like parts of flash messages, or single words for Then steps.

1
2
3
4
5
6
7
8

Scenario: Adding a cat
  Given I'm a user
  When I follow "new cat"
  And I fill in "name" with "Charles"
  And I press "Save cat"
  Then I should see "Charles"

The above feature doesn’t test what it says it does. For example, consider this: Someone tries to add a cat, it is invalid, they see an error that says “Charles couldn’t be added”. This test passes.

Or, someone tries to add a cat, they get no error, but cat isn’t added. However, the word “Charles” is still in the value for the text field. The test still might pass.

In the above feature, the author relied on the coincidence that the cat name would show up on a successfully create. However, since they are testing only that they see the cat name, they are not testing the create at all. Cat adding could be completely screwed, but they wouldn’t know w/o manual testing.

8 – Keep up with the community

Like Rails, Cucumber changes fast. Use blog posts and github to stay current.

Maintainers

Aslak Hellesøy

Joseph Wilk

Ben Mabey

Patch submission & Devel

Cucumber Lighthouse

Current cucumber changelog

The Visitor Pattern

Questions / help

The very active Cucumber / RSpec listhost

Cucumber wiki: Related tutorials and blog posts

Tools

Cucumber wiki: Related tools

Cucumber textmate bundle

Vim cucumber

Email spec – test outbound e-mail

Bonus – Save debug time by looking at the raw HTML

Starting out with Cucumber, I often manually started the rails server and went to the actual page in order to debug non-trivial view problems.

To save myself the time, I wrote a simple step:

1
2
3

Then what

That will print out the raw HTML in a nicely formatted / easy to debug way. While the webrat/rspec matchers have greatly improved since then, I still find myself using Then what fairly often.

Grab it as a plugin here.

Feedback

Do let me know if you have more tips / totally disagree with any of mine.