A simulation to show the importance of backlog prioritization

June 8, 2011 at 6:36 pm | Posted in Agile, Uncategorized | 9 Comments

A successful agile software project depends on a prioritized backlog. You can have a perfectly functioning development team that efficiently finishes one feature after the other, but if these features hold no business value it still is waste.

To show just how important this prioritization is I did a Monte Carlo simulation, using Google Docs spreadsheet. Here are the assumptions I made to create the model:

  • the business value of a user story is randomly generated from a uniform distribution between 1 (very low value) and 100 (very high value).
  • the effort to implement this user story is generated by drawing from a standard set of Scrum poker cards with the values 0, 1/2, 1, 2, 3, 5, 8, 13, 20 and 40 (100 and ? are omitted). The drawing was done first selecting an index based on a normal distribution with average 5 and standard deviation 1. This (zero-based) index is used to select a card from the range of poker cards. For example index 5 corresponds with the fifth card which happens to be 5 as well. Index 6 corresponds with the value 8, etc.
  • the assumption was made that there is no correlation between the business value and the effort needed to implement the corresponding story.
  • the whole backlog can be delivered in 10 sprints.
  • during the first 4 sprints the velocity is increasing in a ration 1, 2, 3, 5. This means that the development team is 5 times as fast in the forth sprint compared to the first. After that the velocity stays constant at a value of 5.

With these assumptions I calculated the delivered business value (as a percentage of the business value of the whole backlog) using 4 different prioritization algorithms:

  • no prioritization. The development team picks up the stories in the (random) order from the top of the backlog.
  • prioritization using the business value of the stories. Stories with high business value are built first.
  • prioritization using the needed effort to build a story. Stories with low effort are built first.
  • prioritization build on the ratio between business value and effort. Stories with a high ratio (high business value, low cost) are built first.

The results can be seen in the following graph:

As you can see from this graph is that not prioritizing your backlog will deliver business value quite late. Since there is no correlation between business value this will be a straight line after the first 3 iterations in which we have a low velocity.

The other extreme is the prioritized backlog using the business value / effort ratio. This is the red line. Here you can see that after 5 iterations you already have delivered 80 % of the business value. For the backlog without prioritization this takes more than 8 iterations!

What looks a bit surprising initially is the difference between the prioritization on business value (the yellow line) and prioritization based on effort (the green line). This is introduced by the fact that in our simulation we have created a bias towards more expensive user stories: the number of user stories with effort lower than 5 is equal to the number of user stories with an effort higher than 5, but remember that the Scrum poker card values are not symmetric: 0, 1/2, 1, 2, 3 versus 8, 13, 20 and 40. That is the reason that in this simulation it is better to prioritize on effort instead of business value.

There is one more interesting aspect to investigate. What is the impact of the standard deviation that we use to draw from our poker cards? I repeated the simulation using the business value / effort ratio for the prioritization and different values for the standard deviation. The result:

The results are easy to explain: when using a higher value for the standard deviation you will get more extremes (either 0 or 40) for the effort to implement a user story. This means that there will be more user stories which require little effort and deliver good business value. These can be build first.

Bottom line: make sure your product owner works hard on a prioritized backlog. Let him quantify the business value of his user stories. As you can see from this simulation half of the effort (and cost) needed to complete the whole project might already be sufficient to fullfil his business needs. This is a huge money saver!


RSS feed for comments on this post. TrackBack URI

  1. actually, and correct me if I am wrong, I see a striking resemblance with a practice I observed often to work flawlessly: prioritize whatever has the lowest estimate before all else. First-time product owners would somehow always come up with this, and … it works almost as good as taking business value into account! Why is that?

    • I think that can only be explained when business value and required effort are not very strongly correlated. In that case you can pick either the lowest estimate with a reasonable chance that the story will have a good business value or the other way around. If the correlation between business value and effort is 100 % then the order in which you pick up user stories doesn’t matter anymore: you always ‘pay’ the same amount of dollars/hours/euros per delivered unit of business value.

  2. Excellent way to visualize this issue. It would be great if you could also include the chance (%) the value will be as estimated to reflect inherant uncertainty in that area.

  3. […] this blog post, Maurits Rijk explains the importance of a prioritized backlog to deliver business value as quickly as possible in a Scrum […]

  4. This may look like a stupid questions, but as I’be been searching for hours now … how exactly do you make a Monte Carlo simulation, using Google Docs spreadsheet? Where did you find the MC script?

  5. Any chance you are sharing the google doc? This is very similar to what I am needing on my current project.

  6. […] Maurits Rijk did a Monte Carlo simulation of a project where implementation is not prioritized (blue) versus prioritized by return on investment (green). Value delivered is the region under the curve. […]

  7. […] Maurits Rijk did a Monte Carlo simulation of a project where implementation is not prioritized (blue) versus prioritized by return on investment (green). Value delivered is the region under the curve. […]

  8. […] item to start first. That this pays off to do so (more or less) right has been shown in blogs of Maurits de Rijk and Jeff […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.
Entries and comments feeds.

%d bloggers like this: