Estimating is Overestimated

Posted by Sergio Romano on Mon, Aug 16, 2010 @ 02:34 PM

This post is from Sergio Romano, Assembla's product leader and skilled technical lead. He's currently deep into the process of building an advanced set of portfolio and ticketing features. Here he takes some time to recommend "iterative planning".

I originally got the idea for this blog article when Santiago Ceria, a professor from my university who also happens to be responsible for process and quality improvement in a big outsourcing company in Argentina, made a bold remark to our class that "Estimation is a REQUIRED step in EVERY project plan". I respected Professor Ceria, but I couldn't agree with him less.

Estimation in Today's World

Software engineering is a young discipline and thus it is heavily influenced by the project management "fashions of the day". You see people jumping between Waterfall, RUP, Scrum, XP or whatever is hot at the moment. Despite the great diversity in these approaches to software development, one can extract some nearly universal principles by threading a middle path between them.

I believe that ESTIMATORS have swung the pendulum too far in one direction and have taken estimation to a point where it is counter-productive. Luckily, there are guys like Andy, our CEO (check his posts about estimation and other things you should avoid), the 37 Signals founders and Kanban gurus who are starting to move pendulum back in the opposite direction.

I see two reasons why estimation is so hyped right now.

Reason 1 - Clients & the Illusion of Control

The first is that clients always ask for a detailed timeline. However, what they actually want is commitment. This carries a lot of weight because when big projects are undertaken, they are generally paid for with someone else's money.

Mostly, they want to trust that you will finish. I may be an idealist, but can't we build trust without a schedule that is managed with a stopwatch? Let's look at an analagous example, do you vote for a presidential candidate because of their timeline or because of the solution they propose to fix a problem? Trust comes from successful incremental releases.

Reason 2 - More detailed plans

The second reason that estimation is so hyped is that estimators often provide a more detailed plan. But the real question is: "Is it always better to have a more detailed plan?"

Let's look at 3 simple examples - each more detailed than the last

1) John will do task A and then he will do task B

2) John will do task A in approximately 2 hours and, after finishing it, he will do task B in aproximately 3 hours.

3) John, who will probably be wearing his favorite yellow shirt, will do task A in aproximately 2 hours and, after finishing it, he will do task B in aproximately 3 hours, but only after taking a 45 minute lunch break.

It's clear from #3 that more detailed plans aren't always better.

On the other hand, the level of detail in plan 2 may seem more reasonable than plan 1. Of course, it depends on the precision of those estimates and the amount of time it takes you to make them. If it took you one day to figure out that it should take John 5 hours to complete his tasks, I bet you would have preferred that he just got started.

Will the more detailed plan make things worse?

I will raise the ante of this discussion, by asking whether detailed estimation will even cause you screw up your project.

Many people will grudgingly admit that initial estimates are often wrong by up to a factor of 4. This lack of precision, whether it is in the form of overestimation or underestimation, is sure to cause problem such as: incorrect decisions, Parkinson's law, less chance of success, etc.

And, if you believe this, we are not just wasting the time we spent estimating but we are actually causing problems in the project itself.Of course, the counter argument is that estimating up front will give you more confidence and control at the beginning of a project. But, is that really the case?

A lot of people think that if you have a very detailed plan you will have more control of your project. I believe that a detailed plan ties your future to your past: You may find yourself saying, "Sure, this is the best way to go, the plan that we created a year ago says so and we haven't had any problems following it so far". That may be the case, but have you consider the new competitors that have sprung up or the changing economic conditions?

The Solution: Iterative Plans

I believe that not only should development of a product be iterative, but also the plan for development of a product as well. This is especially true if you are designing your own products where you have to be very careful not to invest more than is needed.

I think we need stronger theories about iterative plans. Do you want some tips? Check out previous posts from Andy about how we do our work.

An iterative plan is a great choice for an iterative development methodology with client participation (ie: agile). A very detailed initial plan goes against the benefits of a iterative product development. It may be controversial, but I think that minimizing estimation actually minimize the risk of the project.

Tags: agile, project management

COMMENTS

Great post! I totally agree with the need of moving the estimation pendulum back to a useful point.

In most current projects the estimation process is consuming too much resources to achieve an estimation that is never met and could be invested in the project directly from the beginning.

I hope many project leaders and CEOs read this article and change their minds about this problem.

posted @ Monday, August 16, 2010 3:58 PM by Alejandro

Congratulations on your first post @ the Assembla blog, well written piece I must add. I totally agree with the need to leave ideas like BDUP (Big Disign Up Front) behind and focus on stuff really worth our attention, like removing roadblocks and making it easier on the creative/productive people of the team.
Although our project planning at work is light enough so we can spend time on the tasks that really need to get done, I can see how large software companies may struggle to catch on with the notion of iterative planning. I can't help but think of how having a tight schedule gives some PMs the false belief of having things under some kind of control when it's clearly not the case.

Thanks for sharing and keep on writing!

posted @ Monday, August 16, 2010 5:04 PM by Cristian Rasch

Good post Sergio, nice to have more people moving pendulum back in the opposite direction.

posted @ Tuesday, August 17, 2010 7:48 AM by Patricio

Sergio,

Would you agree to an iterative plan when renovating a kitchen in your house? In May your contractor would tell you "$10,000 and I'll be done by July". In June he'll change that to 15,000, an extra plumber on his team, and a due date of September. Now, if you had planned for your in-laws to come in for a visit in August, you'll be in a bind, especially if they have bought airfare already.

I do not argue against iterative planning. However, I think there is a need for a strong disclaimer that there is time and place for iterative planning, when due date is not critical. There is time and place for projects with dates that can not be slipped (i.e. election date, start of school year at a university, shareholder report at a fortune 500, etc). Sometimes even software features can not be scaled down and one is left with having to force overtime to get the job done. I notice that this post and some posts by Andy, while interesting, seem to apply mostly to "designing your own products" as oppose to working within constraints of a large corporation. This is not to say that some of the concepts can not be applied in the corporate world. However, I think that your posts need to be framed accordingly.

Overall, a project manager has to be agile and experienced to utilize best approach for a given project, budget, customer, etc. One also has to be lucky to a degree.

Eugene

posted @ Tuesday, September 07, 2010 2:00 PM by Eugene

Hello Eugene,

Thanks for your comment. I was expecting someone to challenge the content.

If you ask me if I would accept an iterative plan to renovate my kitchen, I'm not sure, I'll probably say yes if it can be split into smaller and significant parts like changing the floor, moving the windows and then painting the walls.

But I will definitely take an iterative plan to renovate my house. Even if I'm expecting visits next month. Why? Because I'll ask this contractor to renovate my kitchen. That's were my in-laws spend most of the time at home while they visit me. Then, we will do another round of planning depending on the time and money left and the results we got working together.

As long as his iterative planning of the work gives me a living (potentially shippable) room, it sounds like a better deal for me. I would invest as much as I want and no more. I will stop when I see that next round of iteration will not give me more significant results for the money I'm spending. And I'll have something better than what I had. But the main discussion here is about planning, not about development, so my main benefit would be not to had spent days agreeing with him about how my ideal house should be and then figured it out that I should get another contractor.

I due agree with you that if due date is critical, you can't take all of the agile practices. But I do think that making a big estimation up front will not give you much more useful data if you already know that your visits will come next month. If you know that due date is critical, just make sure you will have a shippable product in each iteration that will solve the most significant part of the problem. If you do a quick estimation and you see that it does not make sense to start a project with that deadline, you can tell the stakeholders that and offer them to solve what you think is the key problem they are having (or what you think is the best step to get closer to solve it one day). But if you tell me that a rough estimate is not enough and that only a very detail plan will let you know if you will be able to help them and succeed. Then, I'll say that the approach is not good. You are adding a lot of risk by depending on an estimation which you already know will not be as accurate as you are requiring it to be.

posted @ Tuesday, September 07, 2010 2:36 PM by Sergio Romano

Sergio,

No offense, but you seem to be contradicting yourself a bit. In your opening statement you said " Santiago Ceria.... made a bold remark to our class that "Estimation is a REQUIRED step in EVERY project plan"... I couldn't agree with him less"

Then you imply that you will need at least a rough estimate before the start of the project to decide if you should go ahead with it. Even in iterative approach you would need a base estimate up front, won't you? Otherwise, what will you be basing your iterations on?

This contradicts your disagreement with Professor Ceria that ""Estimation is a REQUIRED step in EVERY project plan". In fact, project PLAN is nothing else, but project ESTIMATION.

Now, if Professor Ceria would say that ALL projects should be estimated down to an hour ALL the time, I would understand your dissent from such a statement. However, estimation itself IS required. It is the level of detail that will vary based on your particular circumstances: project budget, length, number of developers, requirements from your customer, etc, etc. Here is where I would agree with you that depending on circumstances, iterative approach could be the best.

Vendors that come to me and try to get business of the company I work for, but do not provide me with a plan of what they will deliver and when, do not see a cent and are put on a list of vendors who waste my time and won't be considered again. If they constantly slip the dates or try to charge much more than the original estimate, the same thing will happen. This is everyday reality. What approach vendors take internally to develop the product is typically of little interest to me. I hope this explains a little where I am coming from, when I say that I agree with your professor in that estimation is required. Repeating myself, I agree with you that the level of detail for that estimation should not be overly granular in order not to hurt the project.

posted @ Tuesday, September 07, 2010 3:23 PM by Eugene

You are not offending me at all. In fact, it's a pleasure for me to be able to discuss it with you as it helps me clarify my point.

I do think you need a rough estimate IF you have a deadline to meet. But not all the projects have a deadline. In many projects, clients actually just want you to commit to work as fast as you can and to help them solve a particular problem. And they will usually confuse that with the need of a deadline. It's our job to show them there are other ways to get commitment and build trust.

I don't think that all plans need an estimation. If you have split the problem into the smallest problems you could think of and ordered them by some priority criteria (that's your plan, planning is making sure you will always be doing the best that can be done in that moment). You can start by just taking the first one in your list and go with that. When you are soon to release or to finish the iteration, move the ones you could not start and stabilize the ones you have started. It takes a bit more to explain, but Kanban methodology uses a similar approach.

As I quickly noted in the blog. Politician work hard to convenience people that they plan is the best without discussing estimations. They propose a way to go in their plans, they try to make us trust that they will be the best to manage it they way we want it to be managed.

However, we agree that sometimes a certain level of estimation is required. That's why the title says: "Estimations is overestimated" and not something like "Estimation is not useful". My main point in this article is that there are many opportunities were people will think that it is needed just because they are used to, or because some old book says so, or because it's the most wide spread way to start a big project.

Anyway, I'm not trying you to accept my point of view. I do understand that in your business, estimation has given you good results that makes it required for you to build a relationship with vendors. However, what I want to show is that there are other people doing good stuff, finishing projects on time and not using estimations as a tool to achieve it. So, we can work on better theories for software engineering, make better experiments and make sure we put estimation in the place it should be.

posted @ Tuesday, September 07, 2010 4:00 PM by Sergio Romano

I think that the "artificial dead line" concept on the comments is a jewel in this post. I always try to convince my clients to work as Sergio suggest, now I have another point asking if they have a real or an artificial dead line.
Gracias Sergio

posted @ Monday, October 11, 2010 10:15 AM by Carlos Peix

Wow Sergio... Why do you have so many comments on your blog and I have almost none? :-).
I said that you need to estimate to build a project plan and I stand by that statement. I don't think that a plan is always needed, or that it's always possible to have a detailed plan. In fact, in most large software development projects trying to build a long term plan is probably a waste of time.
You also know how much I like iterative and incremental development, and that I consider it the right way to develop software. Agile methods like Scrum use estimates to build sprint plans.
I will assume you used my name to get people's attention :-).
BWT, Sergio was a great student.
And a final question... why do you say that you "respected" me? I was hoping that verb would be in present tense :-). Congrats on your blog.

posted @ Wednesday, October 27, 2010 11:07 AM by Santiago Ceria

Comments have been closed for this article.