Why Indeed Do We All Sell Code With Bugs?
In a recent article Eric Sink claims to have found the answer to one of the greatest mysteries of our industry: why do we all sell products with bugs? And the answer is: The alternative is to fix them and risk introducing worse bugs.
OK then. We can now go on with our lives knowing we do good. In fact, by shipping products with bugs, we are saving the world from even worse bugs. Good for us!
If you are a regular reader of this blog, you must already know how much I like such arguments. The kind of arguments that sound so logical. Except that they are missing some important part of the reality we work in.
Let’s start with what’s right with Eric’s argument. It’s true. Not every bug is worth fixing. You have to balance the cost of fixing a bug and the potential impact it might have on you and on your clients.
But this is only part of the picture. Most bugs out there will not pass Eric’s test. Here’s why.
You Cannot Analyze What You Don’t Know
Sure. Some bugs are known at delivery time. You can analyze the cost of fixing these bugs against the risk of living with them (as Eric Sink suggests). However, many bugs are not known at delivery time. They are only revealed after the product is already shipped and used by millions of customers.
Fixing such bugs after the product is shipped will usually introduce a heavier cost and a greater risk than preventing them to begin with (not to mention the potential damage to your clients). Early identification of bugs, relevant education and mentoring for developers, and the use of supportive tools will greatly reduce the risk and the cost of fixing most bugs.
Most Bugs Are Avoidable
Eric’s argument talks about the cost and risk of fixing bugs. I accept that. But you can prevent most bugs. Preventing bugs is not risky (no code change is involved), and in many cases it is cost effective.
Of course, you still have to consider if a certain family of defects is worth preventing (in terms of how much effort should be invested in prevention against the potential risk of releasing the products with such bugs). But in most cases preventing bugs will pay off. A great deal of being a professional is finding ways to prevent bugs effectively.
Sometimes Risk-Cost Analysis Is Used As A Lame Excuse
To be honest, sometimes the risk vs. cost argument is used as an excuse. For some, fixing a bug is always the less convenient option. Adding new functionality is more attractive.
Again, to avoid such a trap, the best strategy is avoid the bug in the first place. This might even prove to be more fun. Did I say fun? I meant worthwhile.
***
If you really want to optimize your development costs try mitigating the risk as soon as possible. Try to prevent bugs, or at least to identify them as soon as possible. This is the cost-effective approach in most cases, especially when you consider the long term benefit.
As for the bugs you are left with just before releasing the product – you should certainly balance the cost and risk of fixing a bug against the severity of the bug and its frequency. Don’t take this analysis lightly, and don’t use it as an excuse for living with bugs that will soon come to hunt you.
***
More information on preventing defects:
May 26th, 2006 at 5:36 am
you don’t qualify or explain “But you can prevent most bugs … [without] code change … [and] cost [effectively]”.
What’s the secret to doing _that_ ?
May 26th, 2006 at 6:27 am
Hi Gavin,
You are right. Thanks for pointing that out.
Please see the update to the post above with the links to other articles.
Lidor
May 26th, 2006 at 3:33 pm
Thank you, Lidor, for your post. That’s what I was thinking when I read Eric Sink’s article.
-TimK
May 31st, 2006 at 10:28 pm
“you can prevent most bugs”
What?!? Are you trying to say you can prevent most bugs just buy hiring a full-time code reviewer? That’s ridiculous. Could you point out any successful software products, whose quality was all based on one dude reading code, that have shipped with only a couple of bugs? Not simple stuff like Tetris games. Real software that’s novel or useful.
May 31st, 2006 at 11:10 pm
Dear Rufus,
This article says nothing about who reads the code, under what contract does he work, and how many hours a week he reads code. Some of the references talk about the idea of professional reviews and mentoring, but all of your assumptions are just assumptions.
Mainly, there is no single person responsible for the quality of the code. This is the joint responsibility of all developers. The question is how to realize this responsibility in the best possible manner. Of course no single person can prevent all bugs. There isn’t any single method of achieving this. But bug prevention is possible using a combination of techniques and the awareness and professioanlism of developers.
Bugs are avoidable! Most of them anyway. Buggy software is not a force of nature. If other industries can create quality products, so can we. The fact that YOU can’t think of a high quality commercial product does NOT suggest that this is not possible.
Unless of course you have already gave up. From your tone (both in this comment and in others) I sense a lot of frustration. Suggesting that professional developers like yourself cannot create software with a much lower bug rate than we are accustomed to is just sad.
Every professional should aspire to create high quality products. That’s true for every craft. Ours is not different.
But don’t take my word for it. Read Steve McConell’s book: Professional Software Development (or any other book on the subject). I guess there are many "naïve people" like myself who believe we (as an industry) can still do a lot better…
And just for the record, I personally know a number of software products which were shipped on time and on budget without a single open bug, and which are still considered bugless at this point of time. Was this achieved by one person? Of course not! This was a joint effort involving effective professional reviews, extensive unit testing, great QC, and of course great developers!
Unfortunately, I can’t provide you with names and references for obvious legal reasons. So instead, I’ll throw in a challenge. If you will contact me personally, I would be happy to discuss this issue with you, and maybe even work with you for a couple of days on your project (free of charge). Maybe we can both learn something from each other.
Waiting for your mail….
Lidor
June 4th, 2006 at 7:15 am
“Sure. Some bugs are known at delivery time. You can analyze the cost of fixing these bugs against the risk of living with them (as Eric Sink suggests). However…”
That’s where you begin talking about something else and yet come to the conclusion Eric is wrong. (?)
June 4th, 2006 at 7:44 am
Yes, because Eric’s argumeant is partial.
What Eric says is true, but it applies only to some of the bugs. Many other bugs are not known at delivery time (although they could have been). I argue that we sell buggy software mainly due to these bugs, and not because we performed a real cost vs. risk analysis.
Lidor
June 4th, 2006 at 9:40 pm
Lidor,
I wrote earlier that you were claiming to be able to get rid of all bugs with a code reviewer. Maybe I jumped to a conclusion there. The reason I suggested that is that the first commenter asked how you propose to prevent most bugs, and you responded with a link to your code reviewing post, and two links to things you’re trying to sell. I didn’t (and still don’t) see how you propose to achieve flawless computer programming.
I would like to know your secret to bug-free software, and I hope you’ll disclose it on your blog. Until you do, I’m not going to believe it.
Thanks but no thanks for the offer of free consulting.
June 4th, 2006 at 9:56 pm
Hi Rufus,
As far as I remember the word flawless wasn’t mentioned in my writings. I never claimed I had a method for creating bug-free software, although (as I’ve mentioned) I know products that are still considered bug-free.
All I said that with the help of some techniques, and extensive professional code and design review among them, you get prevent most bugs.
Think for example of TDD. Can it prevent many bugs? Yes. Does it eliminate Eric’s argument regarding the bugs it prevents? Yes! If developers are not testing their code from day one, you will end up with a list of bugs you have to decide whether to fix or not. With TDD you just have fewer bugs to consider before release.
The same happens with professional code reviews.
I guess the only way to really be convinced is to try it (as well as other techniques of course). I don’t know what’s your bug rate today, and it will probably not be 0 after applying these techniques. But I can assure you it will significantly drop.
I can’t disclose “my secret” because there isn’t any secret. It’s all there: professioalism, awareness, reviews, testing, vision, good project managment. All these concepts/tools (and probably a few others) will make your software better. I don’t have a magic formula, but I know (as I said above) that we all can do A LOT better.
Lidor
August 7th, 2006 at 5:24 am
I’m curious what the supposedly bug-free products are?
I suspect bug-free is in the context of a simple product, or an overly narrow interpretation of the word bug and/or controlled environment.
Personally, I tend to be of the school that you can take steps to reduce bugs, but you can’t completely prevent bugs.
There also no account in the article about time to market, money, or diminishing returns,
What I mean by this, is reducing some bugs can be done with improved methodology and little or no additional cost. Reducing more bugs then starts to cost money and increase time to market. Reducing still more bugs costs still more time and still more time to market. At a certain point it becomes prohibitive (unless you’re NASA or something developing space shuttle software), to reduce still more bugs. Eventually it becomes prohibitive expensive to fix more bugs.
When dealing with unknown at release bugs, you can’t even precisely tell where you are on this curve, but you have to make an estimate.
August 7th, 2006 at 12:12 pm
Hi Sunil,
Please note that my original post talked about “preventing most bugs”. There are two things to notice here:
1. Most is not all. True, as the product becomes larger and more complex, zero bugs is harder to achieve. But I still argue most current bugs are preventable.
2. Prevent doesn’t mean identify and fix. It means prevent. By increasing professionalism and using techniques such as professional code reviews, most bugs are preventable and at a reasonable cost. As a matter of fact, most chances are that with these preventable bugs in your product your development cost will be higher.
Lidor