#Yestimates: The Hidden Benefits of Software Estimation

Software estimation is a notoriously fraught business. Entire books have been written on the topic, including classics like Fred Brooks’s The Mythical Man-Month and Steve McConnell’s Software Estimation. Even with a comprehensive and definitive spec, it is impossible to predict how much time will be spent on obscure details and unexpected hiccups. And how many software projects start with even a half-decent spec?

This frustration has given rise to an increasingly vocal movement proposing to do away with estimates altogether. The appeal is undeniable: no more time wasted on an impossible task that will only give your boss another stick to beat you with. At the same time, there are both obvious and somewhat unexpected reasons why estimates have value even if they don't actually help us to predict delivery schedules. But before we get to that, let's take a quick trip down memory lane.

#NOESTIMATES

Since the dawn of software development, programmers have been subjected to ubiquitous requests for software estimates. No one wants to engage in a quixotic mission, especially if they are going to be held to account when their estimates inevitably turn out to be wrong. The #NoEstimates movement is a natural reaction to this state of affairs.

A blog post by Woody Zuill (and an accompanying tweet) kicked off the movement in 2012. The thesis is simple: if software estimates are always wrong, why even bother? Take all the time you'd waste making useless inaccurate estimates and spend it developing software instead.

This idea isn't all that new. What Zuill describes is similar to kanban-style development, which has been practiced since at least 2004. Kanban practitioners just make a list of stuff they need to get done to deliver their software, put the stuff in order of importance, grab the first thing and start coding. When they're done, they grab the second thing. Keep grabbing things and finishing them and eventually you have a viable product.

The principle is sound. If we accept the premise that our estimates are going to be wrong anyway, any time we spend on them is wasted. What's more, regardless of any disclaimers we might provide, we are likely to be held to account for our estimates and pilloried if – or, rather, when – they prove inaccurate. Whereas if we just forgo estimates completely, we can devote all that time to more productive tasks like writing actual code. And you can't be criticized for inaccurate estimates if you don't provide any.

The #NoEstimates movement has softened its stance over the years. Most practitioners today suggest that rather than estimating stories, we formulate stories of approximately equal size and measure how many stories are completed in a given time period. The provocative name remains, however, and serves an important role in sparking thought and discussion about the role of estimation in software development.

Few people have maintained an absolutist position against software estimates because, despite their drawbacks, there are strong defensible reasons for using them. Some are obvious. Managers, marketers, investors and the like are driven by annoyingly practical considerations like "can I actually afford this project?" and "when will we be ready to launch our marketing campaign?" In the real world, roadmaps and budgets are a necessary evil.

POINTING THE WAY

Some considerations are more subtle. For a long time, we did mandatory story pointing at Salsita on all of our projects. (For the uninitiated, this means that you assign an abstract value to each task representing its relative difficulty without making a priori assumptions about how long it will take. Over time, you start to get a sense of how many story points can be completed in a given time period, typically a sprint of 1-4 weeks.)

Establishing a stable "velocity" (i.e. the average number of points completed during a single sprint) is hard. You need to keep a constant team strength or carefully adjust for any changes (e.g. team members who are sick or on holiday). You need a very strict "Definition of Done" or you might credit story points for a ticket then spend significant additional time down the road fixing bugs, fleshing out edge cases and the like.

But I have observed two major benefits of story pointing independent of their role in sprint planning. First of all, the very process of story pointing forces you to discuss each ticket and make sure everyone understands it. If team members disagree about the right number of points for a story, that is good evidence that it needs to be specified better and/or split into subtasks.

Moreover, the very absence of a stable velocity can be a valuable red flag pointing to problems with your development process. Are we shuffling people around between projects too much to achieve a stable team strength? Are we failing to test enough before labeling tickets as done? Are we assigning points to stories without fleshing them out and understanding the details? Recognizing and addressing these issues can lead to a healthier overall development process.

MUST TRY HARDER

The other hidden advantage of estimates is even more subtle. Even the most conscientious and professional developers code differently when facing a deadline. I've observed this both in my own work and with other developers in my teams.

You might think that a real pro will work through their backlog of tasks at roughly the same pace whether they are under time pressure or not. But with no deadlines looming, there's always time to read an interesting technical article ("it's educational!"), experiment with a new framework ("it might be useful in the future!") or do a deep dive into a feature that could have a much simpler implementation ("you never know!"). This doesn't mean the developers are lazy or sandbagging. It's just human nature.

Healthy, realistic deadlines based on estimates made by the development team itself thus serve an important role in focusing minds. It's hard to do a cost/benefit analysis on how to approach a given feature when you can only see benefits with nary a cost in sight. Committing to a deadline makes the costs and tradeoffs clearer, helping us to make better decisions and ultimately to deliver faster.

COMMITMENT ANXIETY

Everything is bad when taken to an extreme (even licorice). Misuses of estimates, such as forecasts that are misconstrued as commitments, are a widespread problem. In many cases, estimates are made by management without sufficient input from the development team, leading to unrealistic expectations and frustration on both sides.

When done properly, however, estimates can have great value. I recommend using an agile methodology with short iterations that are estimated by the development team using story points. The tickets included in the sprint should be considered a commitment that is communicated to the product owner and taken seriously by the team.

They won't hit their target every time. But the need to make a commitment will help developers to insist that tickets are clearly specified and understood by the team. And the very existence of a deadline—any deadline—will help to focus minds and speed up delivery.