The nonsense of risk managementApril 2, 2012 at 8:36 pm | Posted in Programming | 2 Comments
Let me start this blogpost with a little fictitious story about two project managers:
Paul and Stephen were sitting outside the conference room, waiting for the steering committee to call them in. Paul looked at Stephen with a smug expression on his face. “I’m so glad that I spent that extra money on hiring two extra developers right from the start. That was quite expensive, but since one of our developers has quit during the project and another one got ill for almost a whole month, this was money well spent. We made the deadline! So how about your project?” Stephen sighed. His project hasn’t been that successful. Their test server had failed a couple of months ago and ordering a new one took quite some time. This has caused a severe delay in his project. If only he had invested in buying a backup server. At the project start the team had created a list of risks. They hadn’t forgotten about this one, but the chance of this happening was considered very low. How wrong they were!
The door opened and Paul and Stephen were called in. They sat down. At the other side of the table were five senior managers. Stephen looked very nervous. This was probably going to be his last day since this was a fixed-price project with a fixed date. Right now the company was losing serious money because of the delay. Paul felt very relaxed. One of the managers started. “Well gentlemen, we have studied the results of your projects. And I have to say that one of them has seriously disappointed us. This has been a case of very poor project management, and I’m afraid this will have consequences.” He looked straight at Stephen and paused a few seconds. “Stephen, congratulations on a job well done!”. Next he turned to Paul. “Sorry Paul, I’m afraid we have to let you go. You are fired.” Both project manager’s jaws dropped.
The above story will probably have a surprising outcome to most readers. And why is that? Well obviously because the most successful project manager (Paul) just got fired. And the guy that managed the project that was losing money got a big compliment. They surely must have mixed up the two, right? Well, anyone with some background in project management might point you to the following graph, arguing that you need more information to judge the decision of the steering committee:
As you can see the risk equals the impact (for example in a monetary unit) times the probability. Now every mitigation measure also costs money. The theory is that if the costs of mitigation (for example buying a backup server) are higher than the risk, you should accept that risk. So what happened to poor Paul is that the cost of hiring the two extra developers (the mitigation measure) was higher than the loss he would have had when two team members would be unavailable (due to illness or resignation) times the probability of that event. And apparently Stephen made the right decision since the chance that he would need the backup server was very small.
Ok, now that everyone understands this basic risk management model everything seems very fair and we are done, right? Even Paul might appreciate the decision from the steering committee since he has learned a lot and this will make him a better project manager. Right? Well, almost. There are several reasons why in practice this is never going to work.
Let’s start with one factor in the equation: probability. Probability is difficult beast. For one, it makes only sense with large numbers. If you throw a dice for example, the chance that you will throw a six is one sixth. But you have to throw quite a few times before that number emerges. So how many projects does an average project manager do? For sure not enough to make the law of large numbers work. This means that if a risk in one project actually becomes reality, this is probably not going to be compensated for by other projects in which this doesn’t happen. What’s worse, most people suffer from hindsight bias. In other words, when that test server actually crashes, it becomes extremely difficult to go back in time and think about the correctness of the decision not to buy a backup server. In hindsight the probability has become 1.0 (100 %) instead of 0.000001. And more often than not people will be punished for not mitigating that risk, even though in theory they made the right decision.
Of course most project managers know. So what happens is that they will try to mitigate the risk of getting punished instead of trying to mitigate the risk based on objective calculation over many projects. This makes the whole business of risk management mostly a futile exercise. It will make projects more expensive. Maybe not that single project on which you are working, but for sure over the long run. But it will cover your ass, because unlike the story above in real-life the Pauls of this world will get a bonus while poor Stephen will probably find himself looking for a new job!