Wednesday 8 September 2010

On Improbability

This summer I have read a magnificent piece of literature entitled The Black Swan by empirical skepticist (or was it skeptical empiricist?) Nassim Nicholas Taleb. Its subtitle is "The Impact of the Highly Improbable", and this is exactly what it deals with.

I won't post an exhaustive summary or review of the book, as it has already been done quite well by others, but perhaps a quick introduction is in order for you to follow the rest of this post.

A Black Swan, in this context, is a highly improbably event. Not impossible per se, but an event for which we are completely unprepared because it lies beyond the border of our imagination. 9/11 is a recurring example, or World War 1 - or the discovery of black swans back when man knew only of white ones, for that matter.

The author is a former trader, and so a lot of the reasoning and examples come from the world of finance. The theories are however applicable to most situations - testing, for instance.

Taleb's ideas boil down to a list of concrete advice for minimizing the impact of negative black swan events; making the swans grayer. The gist of it is make yourself more aware of - or at least less intimidated by - the "unknown unknowns", the things that you don't know that you don't know. Yet. And we have all been there, yes? A sprint that doesn't quite go exactly according to plan because we didn't consider every last dependency within the system, we found a showstopper bug just a little too late, two developers were home sick for three days, priorities were changed mid-sprint for this or that reason. But we continue to plan, and we continue to fail.

In my office, I've seen a trend in moving away from detailed time estimates. We used to sit down in the beginning of every sprint, voting for hours on each task and trying to match the available hours. Now, we have an hour-long meeting every week where we go through the backlog and vote for story-points on every new story (and revise our guesses from last week). After a few sprints, we are starting to get an idea of how many points we can churn through in three weeks.

Our morning meetings have moved from crossing off numbers on post-it notes to sharing your "gut feelings" about the tasks at hand. Neighboring teams have implemented a "fist of five" or thumbs up/down voting for their sprint tasks to give the scrum master an idea of how the work is progressing.

Considering the inaccuracies of time estimates, I strongly advocate an approach where less time is spent on guesswork. We are never going to be 100% correct in our time estimates, so let's not put too much effort into them. That hurts less when we're wrong.

How do you do plans and time estimates, and what happens when the unexpected occurs?