(Overly Optimistic) Estimates R Us

Hofstadter's Law: It always takes longer than you expect, even when you take into account Hofstadter's Law. — Douglas Hofstadter

Do projects always take longer than you expected them to? Join the crowd! Apparently this is so common that it's well-known as the Planning Fallacy. As Scott Anthony explains in the Harvard Business Review Blog Network:

The basic concept, first presented by Nobel Laureate Daniel Kahneman and his partner Amos Tversky in an influential 1979 paper, is that human beings are astonishingly bad at estimating how long it will take to complete tasks.

And why are we so bad at estimating? 

In her article on how to be a better judge of time, Heidi Grant Halvorson says there are three reasons we get things wrong.

First, we routinely fail to consider our own past experiences while planning. When my husband tells me it will take him 15 minutes to vacuum the carpets, he is ignoring the fact that it took him an hour to do it last time. ...

Second, we ignore the very real possibility that things won't go as planned. ...

Lastly, we don't think about all the steps or subcomponents that make up the task and consider how long each part of the task will take. When you think about painting a room, you may picture yourself using a roller to quickly slap the paint on the walls and think that it won't take much time at all — neglecting to consider how you'll first have to move or cover the furniture, tape all the fixtures and window frames, do all the edging by hand and so on.

So how do we avoid making those overly optimistic estimates?

If you haven't done anything similar before, following Mike Gunderloy's advice seems wise:

When you’re faced with a large piece of work to estimate, don’t try to come up with a single number to cover the entire job all at once. Break it down into pieces, and then break those pieces down into pieces until the pieces are small enough that you can see how you would do each one and put a number on them.

As a general rule of thumb, if the pieces take more than 4 to 8 hours, they’re not small enough yet. Most people have trouble guessing their time to perform any job that will take longer than that.

But if you have done something even somewhat similar, some people suggest using a different tactic. Here's what Oliver Burkeman wrote about that:

Intuitively, it feels sensible to work out in detail what your projects involve, to break them into chunks and estimate how long each part will take. But the problem with unforeseen delays is you can't foresee them, no matter how finely detailed your planning.

And so, writes Eliezer Yudkowsky on the Oxford University blog OvercomingBias.com, the unlikely trick is to plan in less detail: avoid considering the specifics and simply ask yourself how long it's taken to do roughly similar things before. "You'll get back an answer that sounds hideously long, and clearly reflects no understanding of the special reasons why this task will take less time," he writes. "This answer is true. Deal with it."

Note: You can also read Eliezer Yudkowsky's thoughts on the Planning Fallacy over on LessWrong.


Another Approach: Use a rule of thumb to increase your estimates.

Here's Mike Gunderloy's rule of thumb:

When estimating a task, take your first estimate, double it, and add 50%.

And here's the advice Ira Hyman, Jr. says he was given:

First, make your best estimate, then double that, and finally increase your unit of measurement. If your estimate is one hour, then make it two hours, and finally two days.

Credits: Photo by Mitsy McGoo / scmtngirl, found on Flickr, licensed under Creative Commons

Unison Consulting pointed me to the article by Heidi Grant Halvorson. 99U pointed me to Ira Hyman, Jr. And if you liked what any of the people I've quoted had to say, head on over and read the full articles; it's interesting stuff.