Get Your Teams to Estimate Well

TLDR;

Technologists hate estimating. They know that any estimate they give is wrong, and that makes the team look bad. That is our fault. We ask the team to provide us with estimates that are both precise and accurate. We often ask them to do it using a system of Story Points that has no grounding in reality, doesn't give the individuals on the team enough information to make an assessment, and doesn't provide management enough information to plan properly. Then we ask them to do the same thing that doesn't work over and over again. We are asking them to perform a Cargo Cult ritual that will never bring prosperity.

The solution is three-fold:

1. Throw out the requirement for precision, and focus on accuracy.
2. Train team members in the science and discipline of estimating as taught by How To Measure Anything.
3. Estimate individually, not in a group.

The Point of Planning

Before we talk too much about estimating, let's talk about why we need to estimate. There seems to be an underlying thread in software engineering teams that estimation is a waste of time or something managers want because they think they should have it. When trying to get estimations, you often hear, 'Why? Estimates are never accurate. We will finish it when we finish it.' That answer is dangerous because it is all too often true. However, that's because it's a self-fulling prophecy. If you believe estimates are useless and don't use them to layout your timelines, they will be useless. That doesn't mean estimates are worthless. It means that for estimates to be useful, they have to use them.

The time we spend on some feature or functionality is an investment that we are making. Our resources are not infinite, so we need to manage that investment. You wouldn't start building a house without knowing what it would cost and you. would manage that cost throughout the project. You wouldn't remodel your bathroom if you had no idea whether it would cost ten thousand dollars or ten million. You need to manage your investment in features the same way.

We are not building a house; we are doing something much more complicated. We are making a machine that generates revenue. Each part of that machine has some value that it is going to produce over its lifetime. Our investment in that part should not exceed the value that the part will produce. The numbers are more nebulous than I would like them to be, and different features and functionality can have interesting interactions that change the value proposition. Estimates give us the ability to understand those interactions and react to them. It’s not perfect, but it’s better to have something that is good than not have something that is perfect.

Another reason to have estimates is that it helps us predict the completion time of work. That completion time allows our partners to plan and prepare for features going out. Those estimates will enable us to make deals with external partners and meet the terms of those deals. They allow us to weigh the merits of different approaches in regards to delivery dates.

If we have estimates, we can reduce the load on the team to deliver a particular feature. If we know we need to deliver a feature on a specific date, we have reasonable estimates, and those estimates predict that we will deliver after that date, then we can more easily iterate on the plan. We can remove complexity, simplify features, etc. If we don't have that information, we will miss that date until we get near it, then we don't have those options. Often, the only possibility we have that late in the game is to work more. That is a losing proposition for both the team and the organization.

Dates matter. Some of them are arbitrary, some of them are not, and it's tough to tell which are which. That doesn't matter. What does matter is that when we commit to a date, we meet that commitment. Others in the organization are depending on us, and we are not going to let them down.

What's Wrong With Estimating

Teams hate estimating their work. When they give estimates, those estimates often come back to bite them in the butt. Rarely does the team do a good enough job of defining the work. A story is too often a one-line description with no detail and no definition of done. In too many cases, the team doesn't know the system or have never dug deeply into the Story's area. With all of that uncertainty, we ask them to give us an estimate that is both precise and accurate. We want them to provide us with a single number representing how long the Story will take, and we want them to be right. We might as well ask them to go dowsing for the answer. We would probably get the same quality answer.

'That is what Story Points are for!' I hear you thinking so loudly it almost knocked me out of my seat here in the past. Ron Jeffries invented Story Points to answer this problem. The idea behind Story Points is that you rate the relative size of a Story using imaginary points. Then you look at how long Stories of a specific size take to complete. Over time, you build an idea of how Story Points correlate to real-time, and that will allow you to plan. It's a great idea. Unfortunately, it doesn't work. We are still asking teams to be both accurate and precise. With Story Points, we have added a layer of abstraction that makes it even harder to calculate. Inevitably, the team ends up substituting Story Points for days on a one-on-one basis or assigning points in an ad-hoc manner. In either case, the result is useless for planning.

How to Estimate Correctly

To get estimates, we are going to use a spreadsheet. Why everything devolves into a spreadsheet is one of the great mysteries of life, but they always do. This particular spreadsheet has five columns. A column for the story link. A column for the epic link. A column for a low confidence estimate. A column for the high confidence estimate. A column to indicate if the Story is estimable or not, and finally, a notes field to communicate with whoever manages the process.

We send out a copy of the spreadsheet to every member of the team. Give everyone a couple of days to do the estimation and return the spreadsheet. If there are stories that can't be estimated, those stories are rewritten and expanded until they can be.

The tooling we use expects one number as the estimate for our Story. That is unfortunate, but we aren't going to move the organization to a different tooling set, so we have to live with it. To get that single estimate per Story, we average the p95 estimate and use seventy percent of that as the Story's target estimate. That seventy percent is somewhat arbitrary. As you get practice, you can go up or down in the percentage to more closely map the tooling to your reality.

This process optimizes for three things:

1. Accuracy over precision.
2. A feedback loop that improves story quality.
3. An asynchronous, remote-friendly process.

Accuracy Over Precision

In estimating accuracy and precision are mutually antagonistic goals. You can often be accurate or precise, but seldom both. For example, if I was to predict when my six-year-old daughter's next baby tooth would fall out. I could tell you that it would be next Thursday at 6:34 pm. That is an exact prediction, but it is unlikely to be accurate. If I changed that prediction to be sometime between now and January 1st, 2031, that would probably be accurate, but not very precise. A balance is essential, which is where confidence levels come in. On the low side, we want a confidence level of about five percent, while on the high side, we want a confidence level of approximately ninety-five percent. In other words, you should be about five percent confident you can deliver in the short time frame and about ninety-five percent confident you can deliver in the long time frame.

The book 'How To Measure Anything' by Douglass Hubbard explores this approach. You should invest the time to read that book. It will change the way you look at estimation and general decision making. I recommend that you work through the calibration exercises in chapter five of that book before doing any estimation.

A Feedback Loop That Improves Story Quality

One of this approach's tenets is that risk is represented by how far apart the low and the high number are. The further apart they are, the riskier the estimator thinks the Story is. That gives you useful information about both the engineer and the Story. If a single engineer's estimates are broad, that might indicate that they need more exposure to that area of the code-base. If every engineer's estimates are broad, that means the Story is too ambiguous. That allows you to invest more in the Story. You can iterate that way until the Story is unambiguous.

An Asynchronous, Remote-Friendly Process

It's common to do the estimation with everyone in the room. If you do that, then you get the opportunity to share context, to talk through issues, and resolve problems as a group. Unfortunately, that only works well in face to face communication and produces knowledge that is only available to those that were in the meeting. We live in a remote world, and using processes that optimize for synchronous communication are not as effective as those that optimize for asynchronous communication.

Introducing the New Process

Happy people know what's going on. Write your team's version of this process down and communicate it to the team in a way that allows them to it. You are likely going to get push-back about this process not being Agile. Don't stomp on that push back. Explain why you are doing it, what you expect to get out of it, and its goals. If you recruit instead of command, you get partners in implementing the new process, rather than drag them along.

Using Those Estimates

We have talked about how estimates are useful in general, let's talk about some specific use cases. They let you know when a story is over its estimate.  Once it gets over its estimate by a certain amount, the manager can jump in to figure out what's going on and help the engineer get back on track. Usually, that means getting someone to help them get over some sticking point. Sometimes discoveries are made when you start on the Story, and it needs to be refactored and split up. Regardless, jumping in and addressing those issues helps the team move forward. Having estimates also gives you something concrete that you can use to help monitor that manager's performance. If you see Stories over their estimate by two or more standard deviations, then something needs to be addressed with that manager. You should dive in and figure out what is going on.

On the strategic side, estimates allow you to project completion times and layout work on the calendar. If you are using the right tooling, it also keeps that calendar up to date. When your stakeholders come and ask for new work, you can put that work on the calendar and watch how it affects things. When I can do that, those stakeholder conversations change entirely.

Finally, estimates and good tooling allow you to get rid of status meetings. Suppose your tooling is right and keeps the roadmap up to date automatically. The Stories' status in the tooling is always current, and there is no need to meet to update the status.

Tooling

I use this process in conjunction with Advanced Roadmaps For Jira, which used to be called Portfolio. Advanced Roadmaps give you two capabilities that are critical to your success. The first is the ability to project completion dates for epics and initiatives in-flight based on past performance. The second is the ability to add and remove work that changes those delivery dates without 'saving' it. Those two features together give you a stakeholder management tool that feels like a superpower. The downside is that the learning curve is very steep. If you are using Jira, you should be using Advanced Roadmaps.