On PyCon2013 and Equality

Over the past day, I've been thinking and talking to anyone who will listen about the Adria Richards / PyCon2013 incident, not just because I'm struggling with the overwhelming sense of disappointment in the IT community, but also because I think it's important that we use these incidents to actually get people to challenge what we think we know about the situation with women in IT.

I want to talk as much as we can about this incident - not because it's something that hasn't been going on in the community for ages (it has), but rather because it provides an entry point for us to actually start discussing some of the really important, batshit insane things are are also going on, but are either too personal or too confronting to be documented and publicly discussed.

If you get bored of this post, at the very least, please, read the 4 links down a little bit. They will help to actually give you some real perspective into the real issues I think we need to discuss.

I had no intention of blogging about this, until I saw this tweet tonight :

First British Ruby Conf goes down. Now some dude gets fired. How much more are we gonna take? https://twitter.com/adman65/status/314658755893661696

I had so much to reply to this, but there is no way a pithy 140 character post could contain my outrage - The unspoken division of us and them (because these horrible people keep ruining our conferences and firing our friends); That a dude getting fired is more important than 20% of our community actually feeling safe and comfortable at events; The suggestion that it's outrageous for people to be called out when they are (consciously or not) biassed in their selection criteria.

So here it is.

I've been thinking about feminism and equality a lot over the last 4 months. This started, actually, from a tweet I read following the BritRuby cancellation. Paraphrased, it said that if you were a developer, and looked at your twitter feed and saw nothing but white male faces, you were part of the problem of why women are under-represented as speakers.

This struck a chord with me as I looked at my feed of white, bearded fellows staring back at me. So, I went out and found as many different developers as I could (mainly Rubyists) who would give me a bit more perspective.

Reading the tweets and links from these people, both men and women of different backgrounds, origins and orientations, was simultaneously enlightening and depressing.

As a white, heterosexual male, I've never experienced any kind of discrimination first hand. I've seen some (I live in Australia after all), but you only ever really see the event, never the impact. Some of the links that I have read over the last few months left me amazed and shocked at some of the things which happen frequently, probably in the same conference halls as I have sat, but I, my friends and colleagues have never been aware of (or at least never discussed).

Probably the most confronting posts I've read are these. You should really read them and let them sink in.

These are respected IT professionals. These are members of our community. And their experiences at industry events have included situations which range from uncomfortable to downright freaking dangerous.

This is not acceptable. How much more are we gonna let them take?

(For the record, I don't think this kind of behaviour should be tollerated outside of our community either, but for the purposes of this essay, let's focus on IT).

So this brings us to the PyCon2013 incident.

People are right - the PyCon2013 incident escalated rapidly. What started as a female attendee (Adria) feeling uncomfortable, and asking the conference organisers to uphold the event Code of Conduct, has resulted in both parties losing their jobs, conference organisers receiving harassing phone calls, DDoS attacks on employers, and the attendee receiving death threats citing her home address.

I want to address some of the points in this discussion and share some of the perspective I've gained over the past few months.

Some people have called out Adria's complaint as petty; that she should have just turned around and told the guys to quit it and move on; that involving the committee and taking a photo was unwarranted.

What I have realised from my months of reading is that, as white males, we very very rarely feel in a situation where we are unsafe. I've only really felt physically threatened 3 times in my life, and I can recount to you those incidents in specific detail. They stick with me not only because they were intense moments, but also because they are definitely not the norm - typically, I never need to give my safety a second thought.

For some event attendees (and, indeed, from essays like this, women in many other situations) that pervasive sense of trust and security that I have doesn't exist, or seriously damaged - they may often have empirical evidence from their own lives that suggest that they are, indeed, at risk.

As a white male, I have absolutely no idea what this feels like, or how this tempers one's response and reactions. I think most men are probably in the same situation of having no concept of how this feels.

Sure, Adria could have turned around and asked the guys involved to stop. But what would that have precipitated? Would they have apologised and sheepishly hushed? Would they have argued with her and told her to lighten up? Would they have made lewd propositions to her? Would they have turned to her and screamed in her face and threaten her family? What may have happened to her in the past when she made the decision to confront people in similar situations? Could that have played into this?

It's trivial and safe for us to speculate what might have been the better approach when we have no sense of the risks involved, or how unsafe those situations could be.

The irony is here that most of us have made a similar decision to Adria in different situations - where we've backed down, asked someone more experienced or influential to handle a situation, because, well, it's not worth it and the potential risks are many. If you've ever hit Submit on a complaint form, or marked a tweet as spam, you've done exactly the same thing - extracted yourself from the situation and let a higher authority handle it.

The other item I want to discuss is the notion that she should have just gotten over it; that the joke and conversation wasn't a big deal at all.

First and foremost, this argument assumes that you are in a position to judge what should and shouldn't be offensive to another. This is blatantly naive - we haven't lived their experiences, we don't have their understanding of language, their mindset, and so we can never truly understand the impact our words or actions have on another.

Perhaps the best illustration I've seen of this idea is this Louis C.K. segment about the word "faggot". In it, the point is made that the term "faggot" may be considered hurtful not just because of the intention behind it, but also because of the memories and thoughts that it triggers, causing pain. The words or actions themselves are entirely open to interpretation, that interpretation is what does the harm.

Now, I'm not trying to say that a joke about a dongle is equal to a heavily loaded homophobic slur. But what I am suggesting is that the memories, emotions, and understanding of it could be profoundly different from one person to next.

And to be honest, this is the whole intention of the notion of a professional level of discourse at professional events - because we have no idea at all what might offend others. Codes of Conduct exist not just to enforce the most common areas where people screw up, but to educate them that these areas are likely to cause offense.

Realistically, though, we're all going to mess that one up at some stage. People will interpret things differently to how we intended it, or read far more into it than was intended. All that can be done here is to own up to the fact that, yes, you did just make that person feel uncomfortable or hurt them (hopefully not too badly) and try not to make the mistake again. You can't ever undo the fact that you have caused them pain. People are hard.

The important bit of this incident is not that dongle jokes are bad, or the damage that has been caused. Rather, it is important because it gives us a real opportunity to actually challenge whether our perspective is the only valid one, and start discussing the horrible situation that lies under the surface of all this.

Other things to read:

Addendum:

Do I think the discourse was unprofessional enough to be fired? Probably not. I think that situation was probably poorly handled by his employers. But that isn't Adria's fault - that is the fault of the employer, and the fault of the guys acting the way they were, surrounded by a bunch of people they didn't know. This should have been a learning experience.

Our 3 Laws (of Cucumber)

If you use Cucumber, you need to read Jonas Nicklas' post, You're Cuking it Wrong

It sums up nicely everything I've hated about how most folks write Cucumber.

Seriously - go and read it now if you haven't already… At the risk of ruining my reputation of being witty and charming by using a blogging cliché, "don't worry, I'll wait right here for you to come back".

My eyes were first opened at #rc5 where I saw how others were writing Cucumber scenarios: regexen, CSS and XPath in their features, and overcomplicated Features bogged down in needless repetition of boring steps. Not surprising, these people were unhappy with Cucumber as a solution.

We've been using Cucumber for well over a year on all of our client projects, and we love it to bits (heck, Rik even presented on it at Perth RORO way back in 2009); obviously we use it to exercise our code as our users will, but we also use it to test our APIs, and for testing legacy code that we've just inherited (gak!).

We use a kind of "3 Laws of Cucumber" when writing our features, and this has stood us in good stead:

  1. Does it make complete sense to someone who has never cut code before;
  2. Can it be made shorter;
  3. Can we make it run faster.

(Apologies to Asimov, but you get the idea.)

1) Does it make complete sense to someone who has never cut code before

Our first, and most important goal of writing Cucumber is that the client MUST be able to understand it. Otherwise, why would we bother writing it?

If we want tests which only we can read, we may as well just use rSpec and test everything individually.

However, we consider that tests that the client can read (and more importantly, can help us refine) are essential to our development process, for more reasons than you might think.

Firstly, (and most obviously) they provide a way to confirm new features with stakeholders. They can read the story and mentally make sure that it matches their expectation of what the feature will do. It's at this point where a client can look at the Scenarios and remember "You know what? We need to capture the width of the sprocket here too…". That is, the understandable nature of well written Scenarios allows our customers to imagine what the solution will look like, allowing them to provide us more useful feedback.

Next, well written Scenarios provide a great starting point when we discover there is an error in business logic, because we can put them in front of the stakeholders and say "Right, this is what this process does at the moment. Let's go through it and you yell when something's not right". Sure, you could write up that documentation at the time (or trace the code with them there… gak!), but by virtue of writing our Scenarios in readable English, we already have a high level resource available.

We look at the Scenario, the client can usually spot the problem quickly, we write a new Scenario and get on with development quickly and efficiently.

Finally, by having very little in the way of specialist syntax, it's easier to work with stakeholders to confirm the existing functionality of legacy or inherited code. Often, inherited code comes with all sorts of quirks and convoluted UI pathways which the Customer is already aware of. Rather than us tracing every single link and feature, we can work with our clients to quickly document existing functionality as it sits, before we start making changes. This saves us time and saves the customer money, but most importantly it brings the Customer in to the development process - something which they've probably not had before (almost a guarantee if it's a rescue project!).

In fact, we have found that often, semi-technical stakeholders will quickly take to writing the first cut of these Features themselves, which we then just need to massage to suit steps already available.

To keep our steps readable, we will go out of our way to ensure that there is no XPath, CSS, or URLs in our Features. At the same time though, we are not prepared to trade precision for readability - Then I should see "Welcome Back, Jason" (matching anywhere in the page) is not a suitable substitute for Then I should see "Welcome Back, Jason" in #notice...

Our step would look like this :

To do this, we provide our own selector_for helper (similar to that provided by default for path_to) to map a string to CSS or XPath, and we have steps like :

(You can see the selector_for helper method here )

Obviously, the complexity has to be somewhere, but we will gladly trade easy to read Features for slightly tricky step definitions. (That said, "complex" steps never really seem that complex if you are comfortable with regex - we are).

Update 20Sep2010: Related to this, check out Bodaniel's post about exactly this : Selector-Free Cucumber Scenarios )

We also use complex path matchers when appropriate to keep URLs out of our steps.

To do this, we ensure that every time we refer to a page, it is done so using a human readable name - whether it be a generic page, or a specific one. For example :

In this way, we can easily generate a complex URL given a very simple human description, testing not only the page itself, but also that our routing is working as we expect.

(It also makes it trivial to update our Scenarios if we need to change our routes…)

2) Can it be made shorter

The second rule ensures that we don't create huge, boring scenarios which set up the same state every time; we make heavy use of Background steps, as well as writing "condensed" steps to make the story more succinct.

For example, whilst it may be important for the Authentication process to be clearly defined once (like in an authentication Feature):

When we are doing every other Scenario where login is just something that needs to happen (but isn't the focus of the feature - you know, like most cases) we simply have one step which does this:

This is - as far as I see - one of the major benefits of Cucumber; the ability to create steps of arbitrary complexity to represent complex processes quickly when they don't add any real value to the feature being tested.

3) Can we make it run faster

Finally, we realise that, as Cucumber is testing our entire stack, we need to make performance tweaks wherever we can so as that they are usable during development.

Thankfully, we find that most of our entire Cucumber suites take less than 4 minutes to run on my MBP's 2.4GHz Core 2 Duo. But still, that's long enough for us to avoid running the full suite every cycle, and so instead we only run the features we need (or use autotest) and run the full suite before commit.

(Granted, this is a rather quick suite comparatively… A colleague told me the other day how their test suite took 20 minutes to run… I do not envy them that… That said, their project is easily 10x the complexity of those we deal with day-to-day)

To help reduce the time to test, we make sure that we limit how often we fire up in-browser tests like Selenium, and let Capybara and Webrat do as much of the heavy lifting as we can. Selenium is obviously essential to test AJAX and other JS, but we find that most of the time we can get away without using it if we structure our tests a little more carefully.

In Closing

Cucumber is an awesome piece of software (as are Webrat, Capybara, Selenium, et al) - both as a development tool, and as a tool to help engage and involve non-technical stakeholders in their project.

However, when used incorrectly, Cucumber becomes a truly terrible solution to both.

A poorly written set of Features is verbose, slow to write, difficult to maintain and slow to run. Even worse, poorly written, Cucumber looses the benefits which make it so valuable - it becomes hard for stakeholders to understand, and useless as a resource for facilitating meaningful and clear discussion between technical and non-technical stakeholders.

If you drive a car only in first gear, it's not the car that's useless, it's the driver.

If you're writing garbage Features, it's not Cucumber that sucks - it's the developer.

Facebooker and Cucumber: Within the Canvas and Without

As you may already be aware, Facebooker is an awesome Ruby on Rails plugin which allows you to easily build up Rails applications which interface with Facebook.

I've worked with Facebooker for a few projects now, but I've always managed to avoid needing to test it with Cucumber. Until now.

The whole process was a little non-obvious, especially when you want to write features for pages within a Facebook Canvas as well as for those outside of it (eg. a REST API on your own site, or other, non-Facebook parts of your site).

This post is going to go through setting this up, how to get Cucumber working with Facebooker nicely (to some degree anyway), and some step definitions which I have been using to help make life easier.

One (possibly major) caveat though, I haven't got any Scenarios in place which require redirects, or testing the output of the redirect. For some reason, I keep getting the "You are being redirected" page as Webrat isn't following the 302 correctly. Something to look into.

Anyhu, a bit of an overview of how Facebooker and Cucumber work together.

Overview

When your production app uses Facebooker, a lot of the grunt work to do with authentication, making requests, etc. is handled by Facebooker::Session and Facebooker::Service. Facebooker::Session deals with authenticating you to Facebook, and is used to dispatch a lot of the requests to the Facebook API.

Behind the scenes, Facebooker::Service takes those low level requests and uses different adaptors to make the request to FB over the wire.

When working within Cucumber, obviously we won't have a real session with Facebook and we won't want real requests to be made. Instead, the Facebooker::Mock::Session and Facebooker::Mock::Service classes are used, and these help to stub out the low level calls.

In the case of Facebooker::Mock::Session, this means that you can specify the UID of the Facebook user loading up your page. In the case of Facebooker::Mock::Service, we are able to place XML fixtures under features/support/facebook which will be parsed instead of accessing the network.

Kicking Cucumber until it works

So, how do we kick these into gear in Cucumber?

Most of the heavy lifting is done by simply letting Facebooker set up your Cucumber world, rather than the default Rails world.

This is done simply by changing your features/support/env.rb like so :

When working out how to test with Facebooker, the biggest challenge I had was working out what was and wasn't essential to instantiate the right state for testing - there are blog posts around which have a rather complex step for setting up the environment, but I found that - in reality - all that is required is very simple.

This is the simplest session code I could put together. You can check out a more complete set of steps here (which we're using for our tests).

Nothing all that complex - we open a new session, and set the default for our requests with a few of the common parameters from Facebook. Probably the biggest issue I had here was grokking @integration_session - I'm still not confident I fully understand how deep it runs, but it looks to be some sort of Cucumber magic that sets the session used within Rails.

We can then do something like :

When it comes to actually mapping a model to your Facebook ID, we're using facebooker_authentication - it's simple and does the trick really well.

How to break what should otherwise work

The most important thing to remember is to be very careful if you need to stray from the default code that facebooker_authentication generates, as you can easily break the points where the Facebooker::Mock::Session is passed around. It's not instantly obvious when you make this mistake, but you will begin to see errors complaining about it not being a valid session - that's the Facebook API telling you that your bogus session is, well, bogus!

In our case, we did need to stay from the default - by default .for_facebook_id creates the record as soon as a FB user hits the app, which we didn't want. In our case, the user has to activate their account using an external process (an SMS in our case).

As mentioned above, our initial attempts to replace for_facebook_id were problematic, as we kept getting a real Facebooker::Session, rather than a Facebooker::Mock::Session. Once I worked this out though, the fix was simple - adding a way for us to pass in the already loaded facebook_session object.

Once that was done everything began humming along nicely.

However, in the most common case, for_facebook_id() will do the magic for you.

Canvas, not-Canvas, and everything inbetween

The final issue was getting Facebooker to work for non-canvas pages - pages which are requested outside of the Facebook layout.

In our case, we have a REST API that we wanted to test, but you could feasibly use this same approach to test your front end code with Cucumber if you have it all packaged in the one project.

The biggest issue was that, once Cucumber is set up for Facebooker, it expects all your requests to be using a Facebooker integration session created by open_session.

Thankfully, we can tell Facebooker to forget that we are within a canvas page, and everything begins to work again :

Then, any of your non-FB Scenarios just need to have "Given I am not in Facebook" as a Background step.

In fact, it would even be very easy to set up Tagged Hooks to do this for you automagically.

Vim Tricks for Ruby and Rails Developers

UPDATE - 17 June: My .vimrc file caused my backspace to stop working under gVim. The code snippet has been updated to fix that now.

RailsCamp5 was awesome, but it left me a little depressed. Watching all the folks using their Macs and TextMate, I couldn't help but thing "Hell! Look how super efficient and productive these people are! What kind of second rate environment am I working in!?". It left me a little despondent to think that, regardless of anything else, these folks were going to be more productive than me just due to their setup.

On the Saturday evening, we held a Vim BOF session - a chance for Vim users to share their tips and tricks, and to attempt to convert some of the Mac crowd into the cult that is Vim. I fully expected to go in there and pick up only a few cool key combos that I'd missed, or a few cool syntax files. How wrong I was.

From that session, I now feel entirely better about my dev environment of choice (Gentoo Linux and Vim). In fact, just the things I picked up from that session have me feeling orders of magnitude more productive.

The first awesome tool I had been missing was NERD Tree, which is a plugin for Vim which gives you the project pane / directory style window on the side showing all of the source code in your project. This feature, in fact, was the only reason why I had recently adopted GEdit in preference to Vim when working with Rails apps; it is just so handy to have a list of all of your files available.

To set it up, simply download it (or check out the git repo), and drop the NERD_tree.vim file into your ~/.vim/plugin directory.

To make it easier to access (rather than typing :NERDTreeToggle all the time), I've added a mapping for Ctrl+D to my .vimrc file :

:map <C-d> :execute 'NERDTreeToggle ' .getcwd()<CR>

Then, it's as simple as selecting the file I want, and pressing "t" (to open in a new tab), "o" to open in a new vertical panel, or "i" to open in a new horizontal panel. Combined with "m" (to handle file operations like delete, copy, move, etc.) it's as feature rich as any other project panel.

(Note for those who forget - Ctrl+W and direction to navigate between panels)

The second thing that has helped has been rails.vim, which is a plugin to make Vim more Rails aware. Not only does this fix a bunch of syntax issues, but it also provides some helpers to jump around your project quickly. For example .gf will look for a controller or model which matches whatever is under the cursor. This allows you the quickly jump to the Model from a Controller (for example).

I've yet to play with the integration with script/* and the partial extraction stuff, but it looks cool.

One slight tweak I've made is because I'm under the impression that Rik and I are the only two people in the Ruby community who actually prefer using TABs over spaces in our source code (likely stemming from the fact that I like to tweak my tabs at 2 or 3 spaces, depending on what I'm doing, whereas Rik likes his at 4 or 8). As rails.vim forces the Rails "2 space indentation", I've added a hack to my .vimrc allowing preventing TABs from being expanded.

autocmd User Rails    set no expandtab

Suck it, space-lovers! Do what you like in your projects, I'll do what I like in mine!

And, to make sure I can tell when there are TABs vs. Spaces (something I was missing from e), I have set them to display as characters rather than whitespace :

set list listchars=tab:>-,trail:.
:highlight SpecialKey ctermfg=darkgrey

This means that \t characters are now replaced with a grey >- or >-- depending on my indentation setting (using set ts). It also adds grey "." characters for any trailing spaces at the end of the line - handy for trimming up garbage left when moving code around.

The final tool I've added is the fuzzy_finder plugin, which acts similarly to TextMate's fuzzy finder and allows you to type in parts of a filename and it attempts to find it for you. For example typing "a/m/contr" will search for files matching this sort of structure. In my case, it suggests files including "app/models/contract.rb" - the file I was looking for.

I got most of the information about this from Jamis Buck's blog post Coming home to Vim . Suffice to say, you need fuzzyfinder.vim AND Jamis' fuzzyfinder_textmate code.

Again, I've added some easier keybindings so allow me to use the finder:

" Ctrl+T to find a file and open it in this buffer
:map <C-t> :FuzzyFinderTextMate<CR>
" Ctrl+B to search the buffers currently open
:map <C-b> :FuzzyFinderBuffer<CR>
" Because I don't want to find my log files
let g:fuzzy_ignore_limit = 70

For reference, my complete .vimrc file is :

set ts=2
setlocal spell spelllang=en_au
set noexpandtab

" gvim backspace breaks unless I provide this, so...
set bs=2

" enable syntax by default, ensure we're using a meaningful scheme
colo slate
syn on

" make sure that bottom status bar is running
set ruler
set laststatus=2
set mouse=a
set list listchars=tab:>-,trail:.
:highlight SpecialKey ctermfg=darkgrey

" Stop rails.vim from fucking with my expandtab settings.
" I don't care if everyone else likes spaces, TABs all the way baby!
autocmd User Rails      set noexpandtab

:map <C-n> <ESC>:tabnew<RETURN>

:map <C-t> :FuzzyFinderTextMate<CR>
:map <C-b> :FuzzyFinderBuffer<CR>
:map <C-d> :execute 'NERDTreeToggle ' . getcwd()<CR>
let g:fuzzy_ignore = "*.log" 
let g:fuzzy_matching_limit = 70

These tools, combined with the incredible power that Vim already has (regex substitution, I'm looking at you!), means that I am now feeling more comfortable with my environment of choice. Combined with my other tools - screen, XFCE - I feel that I can get anything done quickly that I need to.

(Reference, gt to swap tabs, or Ctrl+PageUp and Ctrl+PageDown)