If you use Cucumber, you need to read Jonas Nicklas’ post, You’re Cuking it Wrong
It sums up nicely everything I’ve hated about how most folks write Cucumber.
Seriously – go and read it now if you haven’t already… At the risk of ruining my reputation of being witty and charming by using a blogging cliché, “don’t worry, I’ll wait right here for you to come back”.
My eyes were first opened at #rc5 where I saw how others were writing Cucumber scenarios: regexen, CSS and XPath in their features, and overcomplicated Features bogged down in needless repetition of boring steps. Not surprising, these people were unhappy with Cucumber as a solution.
We’ve been using Cucumber for well over a year on all of our client projects, and we love it to bits (heck, Rik even presented on it at Perth RORO way back in 2009); obviously we use it to exercise our code as our users will, but we also use it to test our APIs, and for testing legacy code that we’ve just inherited (gak!).
We use a kind of “3 Laws of Cucumber” when writing our features, and this has stood us in good stead:
- Does it make complete sense to someone who has never cut code before;
- Can it be made shorter;
- Can we make it run faster.
(Apologies to Asimov, but you get the idea.)
1) Does it make complete sense to someone who has never cut code before
Our first, and most important goal of writing Cucumber is that the client MUST be able to understand it. Otherwise, why would we bother writing it?
If we want tests which only we can read, we may as well just use rSpec and test everything individually.
However, we consider that tests that the client can read (and more importantly, can help us refine) are essential to our development process, for more reasons than you might think.
Firstly, (and most obviously) they provide a way to confirm new features with stakeholders. They can read the story and mentally make sure that it matches their expectation of what the feature will do. It’s at this point where a client can look at the Scenarios and remember “You know what? We need to capture the width of the sprocket here too…”. That is, the understandable nature of well written Scenarios allows our customers to imagine what the solution will look like, allowing them to provide us more useful feedback.
Next, well written Scenarios provide a great starting point when we discover there is an error in business logic, because we can put them in front of the stakeholders and say “Right, this is what this process does at the moment. Let’s go through it and you yell when something’s not right”. Sure, you could write up that documentation at the time (or trace the code with them there… gak!), but by virtue of writing our Scenarios in readable English, we already have a high level resource available.
We look at the Scenario, the client can usually spot the problem quickly, we write a new Scenario and get on with development quickly and efficiently.
Finally, by having very little in the way of specialist syntax, it’s easier to work with stakeholders to confirm the existing functionality of legacy or inherited code. Often, inherited code comes with all sorts of quirks and convoluted UI pathways which the Customer is already aware of. Rather than us tracing every single link and feature, we can work with our clients to quickly document existing functionality as it sits, before we start making changes. This saves us time and saves the customer money, but most importantly it brings the Customer in to the development process – something which they’ve probably not had before (almost a guarantee if it’s a rescue project!).
In fact, we have found that often, semi-technical stakeholders will quickly take to writing the first cut of these Features themselves, which we then just need to massage to suit steps already available.
To keep our steps readable, we will go out of our way to ensure that there is no XPath, CSS, or URLs in our Features. At the same time though, we are not prepared to trade precision for readability – Then I should see "Welcome Back, Jason" (matching anywhere in the page) is not a suitable substitute for Then I should see "Welcome Back, Jason" in #notice...
Our step would look like this :
To do this, we provide our own selector_for helper (similar to that provided by default for path_to) to map a string to CSS or XPath, and we have steps like :
(You can see the selector_for helper method here )
Obviously, the complexity has to be somewhere, but we will gladly trade easy to read Features for slightly tricky step definitions. (That said, “complex” steps never really seem that complex if you are comfortable with regex – we are).
We also use complex path matchers when appropriate to keep URLs out of our steps.
To do this, we ensure that every time we refer to a page, it is done so using a human readable name – whether it be a generic page, or a specific one. For example :
In this way, we can easily generate a complex URL given a very simple human description, testing not only the page itself, but also that our routing is working as we expect.
(It also makes it trivial to update our Scenarios if we need to change our routes…)
2) Can it be made shorter
The second rule ensures that we don’t create huge, boring scenarios which set up the same state every time; we make heavy use of Background steps, as well as writing “condensed” steps to make the story more succinct.
For example, whilst it may be important for the Authentication process to be clearly defined once (like in an authentication Feature):
When we are doing every other Scenario where login is just something that needs to happen (but isn’t the focus of the feature – you know, like most cases) we simply have one step which does this:
This is – as far as I see – one of the major benefits of Cucumber; the ability to create steps of arbitrary complexity to represent complex processes quickly when they don’t add any real value to the feature being tested.
3) Can we make it run faster
Finally, we realise that, as Cucumber is testing our entire stack, we need to make performance tweaks wherever we can so as that they are usable during development.
Thankfully, we find that most of our entire Cucumber suites take less than 4 minutes to run on my MBP’s 2.4GHz Core 2 Duo. But still, that’s long enough for us to avoid running the full suite every cycle, and so instead we only run the features we need (or use autotest) and run the full suite before commit.
(Granted, this is a rather quick suite comparatively… A colleague told me the other day how their test suite took 20 minutes to run… I do not envy them that… That said, their project is easily 10x the complexity of those we deal with day-to-day)
To help reduce the time to test, we make sure that we limit how often we fire up in-browser tests like Selenium, and let Capybara and Webrat do as much of the heavy lifting as we can. Selenium is obviously essential to test AJAX and other JS, but we find that most of the time we can get away without using it if we structure our tests a little more carefully.
In Closing
Cucumber is an awesome piece of software (as are Webrat, Capybara, Selenium, et al) – both as a development tool, and as a tool to help engage and involve non-technical stakeholders in their project.
However, when used incorrectly, Cucumber becomes a truly terrible solution to both.
A poorly written set of Features is verbose, slow to write, difficult to maintain and slow to run. Even worse, poorly written, Cucumber looses the benefits which make it so valuable – it becomes hard for stakeholders to understand, and useless as a resource for facilitating meaningful and clear discussion between technical and non-technical stakeholders.
If you drive a car only in first gear, it’s not the car that’s useless, it’s the driver.
If you’re writing garbage Features, it’s not Cucumber that sucks – it’s the developer.
As you may already be aware, Facebooker is an awesome Ruby on Rails plugin which allows you to easily build up Rails applications which interface with Facebook.
I’ve worked with Facebooker for a few projects now, but I’ve always managed to avoid needing to test it with Cucumber. Until now.
The whole process was a little non-obvious, especially when you want to write features for pages within a Facebook Canvas as well as for those outside of it (eg. a REST API on your own site, or other, non-Facebook parts of your site).
This post is going to go through setting this up, how to get Cucumber working with Facebooker nicely (to some degree anyway), and some step definitions which I have been using to help make life easier.
One (possibly major) caveat though, I haven’t got any Scenarios in place which require redirects, or testing the output of the redirect. For some reason, I keep getting the “You are being redirected” page as Webrat isn’t following the 302 correctly. Something to look into.
Anyhu, a bit of an overview of how Facebooker and Cucumber work together.
Overview
When your production app uses Facebooker, a lot of the grunt work to do with authentication, making requests, etc. is handled by Facebooker::Session and Facebooker::Service. Facebooker::Session deals with authenticating you to Facebook, and is used to dispatch a lot of the requests to the Facebook API.
Behind the scenes, Facebooker::Service takes those low level requests and uses different adaptors to make the request to FB over the wire.
When working within Cucumber, obviously we won’t have a real session with Facebook and we won’t want real requests to be made. Instead, the Facebooker::Mock::Session and Facebooker::Mock::Service classes are used, and these help to stub out the low level calls.
In the case of Facebooker::Mock::Session, this means that you can specify the UID of the Facebook user loading up your page. In the case of Facebooker::Mock::Service, we are able to place XML fixtures under features/support/facebook which will be parsed instead of accessing the network.
Kicking Cucumber until it works
So, how do we kick these into gear in Cucumber?
Most of the heavy lifting is done by simply letting Facebooker set up your Cucumber world, rather than the default Rails world.
This is done simply by changing your features/support/env.rb like so :
When working out how to test with Facebooker, the biggest challenge I had was working out what was and wasn’t essential to instantiate the right state for testing – there are blog posts around which have a rather complex step for setting up the environment, but I found that – in reality – all that is required is very simple.
This is the simplest session code I could put together. You can check out a more complete set of steps here (which we’re using for our tests).
Nothing all that complex – we open a new session, and set the default for our requests with a few of the common parameters from Facebook. Probably the biggest issue I had here was grokking @integration_session – I’m still not confident I fully understand how deep it runs, but it looks to be some sort of Cucumber magic that sets the session used within Rails.
We can then do something like :
When it comes to actually mapping a model to your Facebook ID, we’re using facebooker_authentication – it’s simple and does the trick really well.
How to break what should otherwise work
The most important thing to remember is to be very careful if you need to stray from the default code that facebooker_authentication generates, as you can easily break the points where the Facebooker::Mock::Session is passed around. It’s not instantly obvious when you make this mistake, but you will begin to see errors complaining about it not being a valid session – that’s the Facebook API telling you that your bogus session is, well, bogus!
In our case, we did need to stay from the default – by default .for_facebook_id creates the record as soon as a FB user hits the app, which we didn’t want. In our case, the user has to activate their account using an external process (an SMS in our case).
As mentioned above, our initial attempts to replace for_facebook_id were problematic, as we kept getting a real Facebooker::Session, rather than a Facebooker::Mock::Session. Once I worked this out though, the fix was simple – adding a way for us to pass in the already loaded facebook_session object.
Once that was done everything began humming along nicely.
However, in the most common case, for_facebook_id() will do the magic for you.
Canvas, not-Canvas, and everything inbetween
The final issue was getting Facebooker to work for non-canvas pages – pages which are requested outside of the Facebook layout.
In our case, we have a REST API that we wanted to test, but you could feasibly use this same approach to test your front end code with Cucumber if you have it all packaged in the one project.
The biggest issue was that, once Cucumber is set up for Facebooker, it expects all your requests to be using a Facebooker integration session created by open_session.
Thankfully, we can tell Facebooker to forget that we are within a canvas page, and everything begins to work again :
Then, any of your non-FB Scenarios just need to have “Given I am not in Facebook” as a Background step.
In fact, it would even be very easy to set up Tagged Hooks to do this for you automagically.
UPDATE – 17 June: My .vimrc file caused my backspace to stop working under gVim. The code snippet has been updated to fix that now.
RailsCamp5 was awesome, but it left me a little depressed. Watching all the folks using their Macs and TextMate, I couldn’t help but thing “Hell! Look how super efficient and productive these people are! What kind of second rate environment am I working in!?”. It left me a little despondent to think that, regardless of anything else, these folks were going to be more productive than me just due to their setup.
On the Saturday evening, we held a Vim BOF session – a chance for Vim users to share their tips and tricks, and to attempt to convert some of the Mac crowd into the cult that is Vim. I fully expected to go in there and pick up only a few cool key combos that I’d missed, or a few cool syntax files. How wrong I was.
From that session, I now feel entirely better about my dev environment of choice (Gentoo Linux and Vim). In fact, just the things I picked up from that session have me feeling orders of magnitude more productive.
The first awesome tool I had been missing was NERD Tree, which is a plugin for Vim which gives you the project pane / directory style window on the side showing all of the source code in your project. This feature, in fact, was the only reason why I had recently adopted GEdit in preference to Vim when working with Rails apps; it is just so handy to have a list of all of your files available.
To set it up, simply download it (or check out the git repo), and drop the NERD_tree.vim file into your ~/.vim/plugin directory.
To make it easier to access (rather than typing :NERDTreeToggle all the time), I’ve added a mapping for Ctrl+D to my .vimrc file :
:map <C-d> :execute 'NERDTreeToggle ' .getcwd()<CR>
Then, it’s as simple as selecting the file I want, and pressing “t” (to open in a new tab), “o” to open in a new vertical panel, or “i” to open in a new horizontal panel. Combined with “m” (to handle file operations like delete, copy, move, etc.) it’s as feature rich as any other project panel.
(Note for those who forget – Ctrl+W and direction to navigate between panels)
The second thing that has helped has been rails.vim, which is a plugin to make Vim more Rails aware. Not only does this fix a bunch of syntax issues, but it also provides some helpers to jump around your project quickly. For example .gf will look for a controller or model which matches whatever is under the cursor. This allows you the quickly jump to the Model from a Controller (for example).
I’ve yet to play with the integration with script/* and the partial extraction stuff, but it looks cool.
One slight tweak I’ve made is because I’m under the impression that Rik and I are the only two people in the Ruby community who actually prefer using TABs over spaces in our source code (likely stemming from the fact that I like to tweak my tabs at 2 or 3 spaces, depending on what I’m doing, whereas Rik likes his at 4 or 8). As rails.vim forces the Rails “2 space indentation”, I’ve added a hack to my .vimrc allowing preventing TABs from being expanded.
autocmd User Rails set no expandtab
Suck it, space-lovers! Do what you like in your projects, I’ll do what I like in mine!
And, to make sure I can tell when there are TABs vs. Spaces (something I was missing from e), I have set them to display as characters rather than whitespace :
set list listchars=tab:>-,trail:.
:highlight SpecialKey ctermfg=darkgrey
This means that \t characters are now replaced with a grey >- or >-- depending on my indentation setting (using set ts). It also adds grey ”.” characters for any trailing spaces at the end of the line – handy for trimming up garbage left when moving code around.
The final tool I’ve added is the fuzzy_finder plugin, which acts similarly to TextMate’s fuzzy finder and allows you to type in parts of a filename and it attempts to find it for you. For example typing “a/m/contr” will search for files matching this sort of structure. In my case, it suggests files including “app/models/contract.rb” – the file I was looking for.
I got most of the information about this from Jamis Buck’s blog post Coming home to Vim . Suffice to say, you need fuzzyfinder.vim AND Jamis’ fuzzyfinder_textmate code.
Again, I’ve added some easier keybindings so allow me to use the finder:
" Ctrl+T to find a file and open it in this buffer
:map <C-t> :FuzzyFinderTextMate<CR>
" Ctrl+B to search the buffers currently open
:map <C-b> :FuzzyFinderBuffer<CR>
" Because I don't want to find my log files
let g:fuzzy_ignore_limit = 70
For reference, my complete .vimrc file is :
set ts=2
setlocal spell spelllang=en_au
set noexpandtab
" gvim backspace breaks unless I provide this, so...
set bs=2
" enable syntax by default, ensure we're using a meaningful scheme
colo slate
syn on
" make sure that bottom status bar is running
set ruler
set laststatus=2
set mouse=a
set list listchars=tab:>-,trail:.
:highlight SpecialKey ctermfg=darkgrey
" Stop rails.vim from fucking with my expandtab settings.
" I don't care if everyone else likes spaces, TABs all the way baby!
autocmd User Rails set noexpandtab
:map <C-n> <ESC>:tabnew<RETURN>
:map <C-t> :FuzzyFinderTextMate<CR>
:map <C-b> :FuzzyFinderBuffer<CR>
:map <C-d> :execute 'NERDTreeToggle ' . getcwd()<CR>
let g:fuzzy_ignore = "*.log"
let g:fuzzy_matching_limit = 70
These tools, combined with the incredible power that Vim already has (regex substitution, I’m looking at you!), means that I am now feeling more comfortable with my environment of choice. Combined with my other tools – screen, XFCE – I feel that I can get anything done quickly that I need to.
(Reference, gt to swap tabs, or Ctrl+PageUp and Ctrl+PageDown)
Well, it’s been a long time between drinks at this blog. To be honest, that’s partially because not much noteworthy has been happening.
Let me rephrase that. In actual fact, I’ve been busy with lots of interesting things, none of which I’ve had anything interesting to say. So, a brief run-down.
That bottom point is the biggest one, and the reason for this post.
A few years ago when I was working with insilico (or, as it was still known, o’neill software), Paul introduced me to a guy who I immediately got along with. Paul’s comfy office chairs made it far too easy to sit around and chat, and our discussions ranged across a variety of topics – our shared love of Bucky, new technology, business, the environment, and sometimes simply inane conversations about nothing.
Anyhu, fast forward to 2008, and Rik and I started throwing around the idea of coming together to form a business; at the time we were both finding our respective freelance gigs a little isolating, and noticing the problems of flying solo – needing to do everything ourselves, not having colleagues to throw ideas around with, and being overlooked for larger jobs where the risk of a solo developer was just too great.
So, we started working it all out – looking at what we were passionate about, how we wanted to do business, and formulating the very best environment in which two geographically disparate developers could work together closely. We learned a lot in this process – a lot about ourselves, a lot about each other, and a lot about dealing with a 2-3hr time difference across flakey Internet connections. In short, we had a blast.
Finally, almost 3 weeks ago – the 9th of March 2009 – we launched Aurora Software to the world.
Rik and I are both passionate about Ruby, and as such, Aurora Software is focussing on Web Development with Ruby on Rails. Furthermore, we’re openly embracing all of the best practices we love – test driven development, pair programming (a post on this soon, hopefully), keeping things agile, and writing elegant code.
So, that’s my news…
Please head over to the Aurora Software website and check things out.
Super special thanks to George Atherley (of TMIAS fame) who worked with us over the past few months to sort out a kick-ass design for the site and logo. I can only imagine that we were probably the worst client he has had to deal with for a while, but he handled it admirably – he never once told us our ideas were stupid (even when they were), but rather he just smiled and waited for us to realise that, yes, he really did know best!
