Managing Cash at MegaBank

Estimating the Product Backlog

The next step was to estimate how much work would be involved in fulfilling the requirements in the Product Backlog. The team members groaned again, assuming that this task would take forever. They doubted that they could come up with accurate estimates—particularly estimates that were accurate enough to correctly set expectations and guide their selection of Product Backlog at every future Sprint. Before we proceeded with estimating, we discussed the nature of complexity and its impact on software development. To estimate each requirement precisely, we would have to know the exact composition and interaction of the requirement, the technology used to build the requirement, and the skills and mood of the people doing the work. We could potentially spend more time trying to define these attributes and their interactions than we would spend actually transforming the requirement into functionality. Worse yet, even if we did so, the nature of complex problems would ultimately render our efforts worthless. The nature of complex problems is such that very small variations in any aspect of the problem can cause extremely large and unpredictable variations in how the problem manifests itself. So no matter how much time we spent improving the accuracy of our estimates, the estimates would still be wildly inaccurate.

After we’d had this discussion, I asked Julie and the team to take a crack at the estimates, bearing in mind the following guideline: the purpose of estimating is to get a handle on the size of each requirement, both in its own right and relative to the size of the other requirements. This information would help us prioritize the Product Backlog and divide it into Sprints. I reminded them that Scrum is empirical and ultimately based on the “art of the possible.” The team had only to do its best during each Sprint, and we would then update our expectations about what could be done by the end of each Sprint. We would track actual progress on each Sprint’s Product Backlog to determine and project progress on the project as a whole. From this projection, we could predict when the system would be ready for release. In this case, management expected the system to be ready in five months. We would now take a first stab at determining whether this was a realistic expectation. At the end of every Sprint, we would update the expectations by tracking actual delivery of functionality against expected delivery of functionality.

With these guidelines in mind, the team was able to estimate all of the Product Backlog in just one hour. The team based its estimates on its knowledge of the current mainframe cash application—which all the team members had worked on—and team members’ understanding of J2EE, Java, and the new technologies that they would be employing. The team, Julie, Tom, and Ed were eager to see how these estimates compared with management expectations: did these estimates indicate that the project would be done in five months? Ed was particularly interested because he was the one who had predicted as much.

What Does “Done” Mean?

Before we proceeded, I asked the team what information its estimates included. Did the estimates take into account how long it would take to analyze, design, and code the requirements in the Product Backlog? Did they include time for unit testing? Were the unit tests automated on something like JUnit? Did the estimates allow time for code reviews, for refactoring, for writing code cleanly and legibly, and for removing unnecessary code? It was important that everyone understood exactly what the estimates allowed for because understanding would prevent people from thinking work was “done” before all of the work taken into account in the estimate was complete. Julie wanted to know why I wanted to discuss this. I told her that the functionality being developed would be more or less valuable according to what work the team performed before considering a requirement fulfilled. For instance, if the team didn’t unit test the functionality, we should probably count on finding more bugs later on. If this were the case, we ought to allocate more time for testing of the application prior to implementation because we’d have more bugs to fix. Similarly, if the team was refactoring code as it worked, it would be easier to fix bugs in the future—and the application would be easier to maintain and enhance.

Julie wasn’t familiar with JUnit; one of the team members told her that it was an automated testing facility on which the team could build a suite of automated tests to run against the application. Whenever a new piece of code was added, the team would immediately know whether it broke any other pieces of functionality. Julie was fascinated. She wanted a tested and maintainable application, not something that was coded too quickly. She had always assumed that this was what was being delivered and was glad that she now had an opportunity to let the team know that was what she expected. I asked the team to now reestimate all of the Product Backlog with this new knowledge of Julie’s expectations. After spending an hour figuring out the impact of this new definition of “done,” the team had updated the estimates. Julie then discussed the Product Backlog with the team. Which requirements should be tackled first? Because QA wasn’t part of the team, what work could be completed as a unit and given to QA to start testing at the end of every Sprint? What nonfunctional requirements had to be prioritized? The result of this collaboration was the prioritized Product Backlog.

How Hard It Is to Change

It was time to plan what the Team would do over the course of the first Sprint and subsequent Sprints. We estimated how much time on average the team members had available every month. We added this up to get a rough feel for how much time the Team could devote to each Sprint. Then, starting at the top of the Product Backlog, we identified how many items could be potentially included in the first Sprint. We continued down the Product Backlog, estimating potential Sprint contents until the entire Product Backlog was parsed into seven Sprints.

We all sat back. Ed had promised that the Team would deliver the system in five months. Our rough calculations indicated the system would be ready in seven months. Nobody said it, but we all knew that the new definition of “done” contributed to the additional two months. If we had stuck with the Team’s first estimates, we might have ended up with an estimate of five months. And if the Team hadn’t worked according to this new definition of “done,” it might well have delivered the system in five months. But because Julie now understood what “done” meant, the extra time was required. I looked at Julie and asked her whether she wanted us to go back to the old estimates. Julie was upset. She wanted to know how we had committed to five months if we knew that we would be delivering a substandard system. I told Julie that we hadn’t known, that until this planning session we couldn’t actually predict one way or another.

However, Ed had agreed with his management that the project would be done in five months. Now he would have to tell his management that he had been wrong. I told Ed that this shouldn’t be a problem. After all, it was Julie who was paying for the system, and she understood why the estimate had increased from five to seven months. Also, for all we knew, the Team might finish in less than five months—or more than five months. At this point, we couldn’t be certain. We would know a little better by the end of the first Sprint; at that point, we would have an idea of how quickly the Team could turn Product Backlog into functionality, and we could adjust the estimated number of Sprints. Alternatively, if we wanted to increase the speed at which the Team was working, we could bring some people who knew the old cash system on board. These were all possibilities that Julie, the Team, and Ed could consider at the end of each Sprint.

Ed was profoundly uncomfortable with this approach. In the past, he’d always stuck to his initial estimates, and the Team had never let him down. Yes, he agreed, he now had better information about the project than he had before. However, the culture at MegaBank was such that once you said five months, that was all anyone remembered. Ed then turned to the Team and said, “Look, I know we have better information now, but even now it is still an estimate. We have five months, you’ve never let me down before, and I’m going to count on you not to let me down now.”

A profound silence followed Ed’s declaration. One team member later told me that it had sounded to him as though Ed were saying that it was business as usual, Scrum or no Scrum. They might be doing iterative, incremental development, but they were still going to cut corners whenever necessary. Ed was unwilling to tell his managers that software development is complex and that any estimate is just that—an estimate. Instead, the culture of believing that one can predict the future and that there is never a need to adjust predictions would continue to prevail at MegaBank. What the planning meeting had exposed was that until now the Team had been cutting corners to prop up this facade. Julie had heard Ed tell the Team that the date was more important than the quality, and that they should do whatever was needed to meet the date, even though Julie had asked for a quality product.