Fixed price projects? No, thank you!

November 18, 2010 at 9:23 pm | Posted in Programming, Ramblings | 18 Comments

In IT it is quite customary for a customer to ask several suppliers for a bid on a project. Often he wants a fixed-price bid, which means that functionality, quality and price are all fixed. More often than not there is also a deadline. The rationale behind this approach is that the customer wants to transfer the project risk to his suppliers. Of course this doesn’t work out in reality.

There are many other reasons to dislike this approach. For one, it puts the customer and supplier in an opposite position right from the start. This later on shows in struggles about bugs versus features, different interpretations of the requirements, high costs for even the tiniest change request, etc. To be short: I have never seen this work in the 20 years I have been involved in IT. At best you have a win-lose situation, most of the time it results in a lose-lose for both supplier and customer.

 

In this blog I will not go into the ethics of fixed-price contracts. Scott Ambler wrote a nice article about that in Dr. Dobbs back in 2007. Instead I will present the results of a simple statistical model. These results will show that right from the start a fixed-price project is set to fail.

This model is based on the following assumptions:

  1. Estimating is difficult. At the start of a project it is easy to be wrong as much as 50 %. Based on accurate requirements that don’t change (!) it is possible for trained estimators to come up with an estimate that is 20 % accurate at the most. And this is only when they can estimate many similar projects.
  2. The second assumption is that suppliers over- and underestimate around the real effort that is needed to finish the project. We have modeled this as a normal distribution.
  3. If there is more than 1 bid, the customer will select the lowest bid. This may sound short-sighted. And it actually is, but it is also reality.
  4. Supply and demand are balanced. This means that suppliers don’t have to offer a lower price when there isn’t enough demand. It also means that they can’t ask high prices for the same reason.
  5. All suppliers have the same productivity and hourly rate.

I created a Google docs spreadsheet which generates 500 test-runs with each 10 suppliers. Next I calculated the minimum for every test-run for 1 supplier, 2 supplier, etc. until 10 suppliers. Finally I averaged all these minimums to get a reliable number.

An example: the real amount of work was set by definition to 100. When selecting a standard deviation of 10, an average run of [100 95 92 90 89 99 87 86 85 84] is generated. This means that with 1 supplier on the average the bid will be 100 % of the real effort. With 2 suppliers, the chosen bid will already be 5 % too low, and so on.

The results for standard deviations ranging from 0 to 50 are shown in next graph:

What immediately becomes visible is how bad the situation actually is. Take a typical case where 5 suppliers compete for the same project. Combine this with a standard deviation of 35 (remember the mean is 100). In that case the bid that wins offers to do the project for 60! In other words, 40 % lower than the actual price. No wonder he will end up fighting with his customer after a couple of months.

In my next blog I will drop the assumption that demand and supply are balanced. I will show that an unbalance in favor of the suppliers will have a devastating effect. I will also show that suppliers hardly profit from the opposite situation.

Remember, if anyone asks you to do a fixed-price project: just say no :)

About these ads

18 Comments »

RSS feed for comments on this post. TrackBack URI

  1. What happens when you drop the assumption of supplier equivalence?

    • Of course that can have a big impact. If you got a team of brilliant developers you will have a higher speed and will therefor be able to offer a lower bid. On the other hand, if you have such a team, do you really want to give away a potentially large profit margin because your competitors are so bad at estimating?

      I added this assumption because there are (by definition) not that many brilliant teams/companies out there. So you as a customer you will get mostly bids from average suppliers with average developers.

      I could improve my simulation by taking differences in suppliers into account, assuming that this is also some kind of normal distribution. In my next blog I will first drop the assumption that supply and demand are balanced.

      Thanks for your suggestion!

  2. “if anyone asks you to do a fixed-price project: just say no” – that’s a fine thought if you exist in a make-believe world. The reality is we need to work to make money to pay our bills. And for many freelancers, turning away any/all fixed-bid work would mean going broke.

  3. Gosh, I’m surprised by your conclusions. I’ve been doing fixed-price projects for 20+ years and in all that time, only one project has ever run over budget – and that one, not painfully so.

    Projects sometimes overrun estimated time frames due to delays with customer and third party deliverables, but if one has several projects running in parallel, it’s possible to work around these delays with only occasional hiccups in productivity.

    Mind you, I don’t do any work in the public sector ;-)

    • I envy you :)

      Just to make the context clear: I’m not talking about small projects were a single productive developer can make a huge difference. This is about significant projects for which the customer asks several suppliers to make an offer.

      At the end a lot of fixed-price projects stay within budget for the simple reason that some of the requirements are ‘adjusted’, because of commercial concessions or because the supplier asks a lot of extra money for a change. I have also seen projects were the supplier accepts a loss during development, knowing he will earn it back during maintenance.

      However, as soon as any of this happens the relationship between the customer and supplier will almost always take a severe hit.

  4. I cant believe you have spent 20 years in IT! When you order a laptop with some custom config, do you pay a fixed or variable price? When you order a new piece of hardware directly from say Intel, wont you pay a fixed price?

    Why should software be treated any different? I agree exact estimation is tough. But surely any kid can figure out what the hard limit would be for a requirement. When a firm quotes a price they usually have a buffer for protection. Sometimes, they have to come down and take a loss just to keep the business going and establish good relationship with the clients. In the same way, certain firms choose vendors based on experience and not just based on the lowest quote.

    Glad you made it to reddit. But surely you have failed on your IT experience. Better choose a better career that suits you.

    • Clearly you miss the distinction between project and product. IMO you can do a product fixed price since you know what is asked. Software projects are more like a “wicked problem” http://poppendieck.blogspot.com/2002/01/wicked-problems.html . You know what you want when you get there. As long as projects have this “wickedness” there is no fixed price.

  5. Prabhu, clearly you have no experience in software development at all, if you compare the development of software with the buying of a laptop. Please, after you finish school, get some real experience before posting such nonsense as a reply to a well thought-out blogpost.

  6. Erik,

    You may be right that you can’t change the world when you’re a free-lancer in a market where fixed-price is the only option.
    But why do you think that agile is so popular nowadays, in companies ranging from the tiny to large institutional banks? Because we know now (or we’ve known for 30 years, but management accepts it as true) that the development of real-world software applications – at least business software – is so fraught with uncertainties in so many areas (starting primarily with requirements and which ones are really needed) that any up-front planning is doomed to fail and will never resemble the final product. If you accept that – and many people do nowadays – you must also accept that the traditional fixed-price project can’t work either. Fixed price can only work if scope is not fixed, and that’s what in reality always happens – in almost any successful fixed-price project, the cope was modified during the project.
    People who don’t believe that should look at an actual example and dig a little deeper; they’ll find out that it’s true.

  7. The organisation I work for tends to do fixed price contracts with the (large) software developers we use. And the reason we do that is to incentivise them to be efficient (e.g. not pad out their projects with unnecessary bureaucracy and makework).

    You’ve assumed this isn’t an issue i.e. all suppliers have the same productivity. It’s not surprising that if your analysis assumes away the main factor that drives people to use fixed price contracts, then fixed price contracts don’t look like a good idea.

    • I totally agree with this sentiment. In the world I’ve lived in, the only way to force people to make good and cost-effective decisions is to provide a bit of ‘pain’. A fixed price contract provides this.

      It forces those asking for things to really weight cost/benefits of their requirement changes and decisions. Anyone can always ask for feature X, because why wouldn’t you. Austerity drives creativity and new ideas.

      The other reality is that you rarely know what you’re building until you’ve built it. But to solve that you don’t give people open-ended cost+ contracts. You give small, fixed price contracts that only get you to the ‘next step’ in the process. Build out features X,Y,Z for some number of dollars. Then lets talk again, iterate, and pay for another step.

      Something like that (I think) solves both issues.

      It forces the buyer to plan in incremental steps, not thinking they know everything. Protects both parties from feature-creep (since the amount of work you can get done is fixed by price). But doesn’t ignore the reality that things cannot be estimated accurately. It keeps the incentives in the right place for buyers to be realistic with requirements (no gold plating). It keeps the developers honest and incentivized to be efficient (since they only get so much $$ for a feature)..

    • This might be news to you, but the developers you’re using are a lot more inefficient than those using better (agile) models. Fixed price forces your suppliers to take your requirements at face value, and build the simplest solution that can be pass them, instead of the simplest solution that works. It also forces a short term view on them: copy and paste code is not a problem for them, only for the guy with the next (maintenance) contract.

  8. I love the funny comments of these cheap RentACoder Indians :)

  9. It’s an interesting argument, but there is a counterpoint to offer. You are correct when you say that fixed pricing removes risk from the customer (which is why the customer wants fixed pricing), but the risk of inaccurate estimates is not the only thing in play. From the customer’s point of view, they’ve budgeted a certain amount (X), based on your quote. If your project overruns the budget then there might simply not be the money available to pay the new price.

    Secondly, fixed pricing removes the incentive to under-estimate the cost to get the contract then run over budget for the extra cash. Now, I’m sure everyone here is honest and forthright, but not everyone is, so a customer would be reluctant to hire someone who would not agree to a fixed price, possibly smelling something fishy. Personally, I wouldn’t hire a builder unless I had a reasonably fixed price (and in terms of predictability of estimates and implications of running over budget, the two industries aren’t that different).

    Finally, the project may no longer be worth the price, and an alternative (such as hiring extra staff to do whatever the project output was going to be doing).

    Up-front estimates are notoriously unreliable, and it costs the software industry (particularly the consulting industry) a lot of money when they’re wrong (which is a lot). However there’s other things to consider, and there’s valid arguments for fixed pricing other than just passing those costs onto the supplier.

    It’s possible to mitigate some of these issues by including a deviation clause in the contract to allow for some leeway, or thinking about why the estimate was wrong. Was the spec from the customer unclear? Did the customer change the spec partway through? If so, charge them for it (but make sure they know about those charges up front, and maybe they won’t change the spec).

  10. Well, I have been programming for a living for over 30 years now, and I can tell one thing: Estimates only work when you can reuse the assumptions: same team, same technology/language, same kind of software/system being developed, etc. Guess what, nobody wants those client/server desktop apps I used to develop some 15 years ago, nobody wants that dull WEB 1 sites/intranet apps, from 5 years ago, technology changes steadily… Nobody wants a basic CRUD + Reports app anymore, apps need to be SEO-friendly/User-Customizable/Social/etc… I’ve worked with dynamic teams over many companies/contractors, so I hardly worked in more than 2 consecutive projects with the same people… So my conclusion is unless you really found a niche (probably on Software Maintenance) things change fast enough in all those dimensions that estimating is truly a very risky exercise, so I agree with Maurits, ‘Fixed Price Projects’ are a ‘no, thanks’ proposition for me.

  11. Apparently still a lot of people still love to compare software engineering to building a house. This comparison oversimplifies what software engineering really is. Just think of all the differences between software and a house.

    However, a later thought the comparison with construction holds to some degree: renovation.

    Let’s say you’ve inherited an century old house from an old-uncle who has lived there for 50 years. Nothing was improved for the last 40 years so there’s certainly a lot to renovate. Some walls might even be about to collapse, the house could contain materials that in current time can only be handled by expensive specialist. On the other hand, in the attic you there might be extensive paintings or antique hidden.

    The house is also a official monument, so you can’t just tear it down and rebuild something. Finally, the house is at a top location in the center of Amsterdam. So you can be very certain you can sell or rent of for a very high price once the renovation is finished.

    You know quite well what you like and dislike in a house. You’re pretty certain you want the kitchen down stairs and the attic could be a very good apartment for a au pair. And it looks like creating a jacuzzi at the second floor should be easy.

    Could you do a renovation project like that for a fixed price, or would it be better to take on the project on a day by day or week by week planning, and a vision where you want to be over a year?

    In my opinion all software projects are like the above. You might know what you want in the end, but it’s impossible to know exactly how to get there.

  12. […] Again I partly succeeded. The average number of visitors this year was 71 a day. My blog on the evilness of fixed-price projects attracted almost 6000 visitors in a single day. My lesson learned is that I really have to blog more […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com. | The Pool Theme.
Entries and comments feeds.

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: