The Emperor’s New Clothes Syndrome
There’s a huge gap between how software development is discussed in books, articles, blogs, and forums, and the way software companies actually run their R&D operations. It’s as if we live in two parallel universes. In one universe most of us know exactly what we should do to develop better software – software that has more value both to our business and to the end-user — and do so on time and on budget. But somehow, when we put down our books and logout from our online world, we go back to the parallel universe of corporate business, where most projects are either delayed or canceled, and most software products disappoint their users.
When asked, many business leaders will tell you that books, articles, and Web discussions, are all theory — no one can apply them in the real world. Reviews, entry and exit criteria, happy and satisfied workers, creativity, unit testing, giving up features for sane schedules, mentoring – all are infeasible under the constraints and forces of the “real world”, so they say. And that’s a really smart argument, as you will see in a second.
So, how come so many smart and talented people are preaching for years for better software development and are being read by so many people and still so many businesses are making the same mistakes over and over again. I think the answer lies in the Emperor’s New Clothes Syndrome.
Deep inside, many of us know exactly what needs to be done to improve the way we develop software. A couple of years ago, a student approached me where I worked and “naively” asked me why we are not doing code reviews for new code written. He knew exactly what needed to be done. And if he knew it, you can bet almost all other developers in our department knew it. And still…
The problem is that although we know exactly what doesn’t work right and how it should be fixed, most of us will never say anything. We don’t say anything because there’s a very good chance the minute we do we will be marked as uncooperative, pessimistic, or simply detached from the business reality.
That’s exactly why everyone was afraid to admit the Emperor was naked in Hans Christian Andersen’s tale. The premise that only smart people are able to see the Emperor’s clothes is one of those “perfect arguments” that can never be disproved. The only one who ultimately said something was a little child who was naive enough to ignore the implications of his actions. Luckily, with his naive reaction this child immediately freed everyone else from the paralyzing power of the premise. Things are not that simple in our business world. If you are marked as a troublemaker, the rest will rarely follow.
And the exact same syndrome sometimes afflicts us as an industry. Much of the hypes and trends that are sweeping the software industry are far from being perfect as they are presented to be. They come out of the box with the premise that anyone who doesn’t accept them knows nothing about modern software development. And thus, instead of being engaged in an intellectual debate, we all get emotionally involved in a lost battle.
I guess the only way out of the Emperor’s New Clothes Syndrome is to nurture our professional integrity and keep saying what’s on our mind. At the end of the day, if more of us do so, the business each of us work for, and our industry as a whole, will become better.
If more of us say what we know in our hearts to be true, the rest won’t be able to ignore it anymore. Hopefully.
August 27th, 2006 at 8:08 am
Candor is one of the most important things to have in a business. Jack Welch talks alot about this and I’m really really glad to see the business world and the computing world converging around this topic.
August 27th, 2006 at 10:47 am
[…] I hate to keep pointing out one person’s posts, but Lidor is a great writer and says stuff so much more eloquently then I ever could. This time he writes about how we are all afraid to speak up and say what is really in our hearts about how we do software development. We know we should be trying something different, or stopping some technique that is more cost than benefit, but we just go along with what everyone else says, afraid of being labeled “uncooperative, pessimistic, or simply detached from the business reality”, or worse. When you see a trendy technique that everyone thinks is the silver bullet, but somehow you just don’t see it, speak up. Be that naive little boy who doesn’t know any better and point out that the emperor is naked. […]
August 27th, 2006 at 11:43 am
[…] The Emperor’s New Clothes Syndrome: “The problem is that although we know exactly what doesn’t work right and how it should be fixed, most of us will never say anything. We don’t say anything because there’s a very good chance the minute we do we will be marked as uncooperative, pessimistic, or simply detached from the business reality.” […]
August 27th, 2006 at 1:13 pm
[…] Lidor Wyssocky at The Mindset writes about “The Emperor’s New Clothes Syndrome.” This reminded me of a talk Tim Lister gave earlier this year at the Boston SPIN. “The problem is that although we know exactly what doesn’t work right and how it should be fixed, most of us will never say anything,” Lidor writes. […]
August 28th, 2006 at 9:45 am
I think part of the problem is that - with the exception of the new employees - everyone within an organization has donned some of the clothes themselves, too. So we get doubly embarassed. We get comfortable with the status quo, even though we know it’s going to hurt us in the long run. So it’s more than just the stigma of being the whistle-blower.
After all, code reviews are a pain in the butt. They impact schedule, they are rarely at “convenient” times (for the reviewer or the reviewee), and they “feel” less like progress than hammering out your next task. We all know of the benefits by now - many of us from first-hand experience - but it’s too easy to stick with the same old habits. Same for many other practices that have been proven to work.
August 29th, 2006 at 5:51 am
Certainly there are two parallel universes, we all know what’s right and what’s wrong yet we repeat the same decisions/mistakes year after year.
Point here is companies don’t wake up unless it hits them badly (financially of course), till that time 5 layers of cream enjoys the fruits of initial success and the bottom layers is crushed under pressure, sometimes working for 18 hours continuously to deliver features.
Lack of process, documentation, politics, etc makes matters worse for the poor developers. It’s appalling to hear negative feedback during the appraisal review just because developers raised concerns over lack of process and documentation.
These companies will survive initially but if they don’t change or rather I should say if the people don’t ask for change then company will have hard time to face the competition.
I hope things change……. :)
August 29th, 2006 at 7:05 am
[…]Lidor Wyssocky has some good thoughts on why it is that developers don’t implement changes that they know would be helpful.[…]
[…] [T]here exists both a flip-side and a problem with Lidor’s points that are inherent in professional integrity that need to be explored.[…]
August 29th, 2006 at 10:51 am
Hi Venkat,
Your analysis is sooooooo accurate! I am having the exact same thoughts.
The people who are most affected by a bad process (or other problems) are not the one in control. The one in control are usually not immediately affected by the situation.
But to run a successful business you have to look for the long run. And in the long run, everyone will be affected in one way or the other.
Thanks for a great comment… I am certainly going to write about this pyramid effect sometime soon.
Lidor
August 29th, 2006 at 11:18 am
True that the “ones in control” determine what happens.
My experience does not necessarily support the “burning boat” idea, though. The idea that if things get bad enough, people at the top are affected, and then change. Sometimes they don’t. They stay the course right to the fiery end, or they bail out early. (There are exceptions — but leaders who change, in my experience, change through their own personal development, not necessarily due to anything happening around them visible to others.)
Rather, I think the “man makes history” idea is more applicable. A company or a department will go along in a certain way until, perhaps due to crisis or failure, or perhaps just due to someone moving on to another job, a leader arrives who starts doing the right things.
Of course, for people not at the top who want to speed up such changes, you’re back to pointing out what people don’t want to hear: that in a given case, the current leader is simply not right for the job. Of course, you have to be sure that somebody has a better leader available.
March 17th, 2007 at 2:04 pm
This is completely correct. You hit in right on the head. I was thinking if I was the only person thinking like this, but I’m glad you brought it into the light. Very nice post.
January 9th, 2008 at 1:26 pm
[…] Lidor Wyssocky has some good thoughts on why it is that developers don’t implement changes that they know would be helpful. […]
January 9th, 2008 at 3:16 pm
[…] Professional Integrity Lidor Wyssocky has some good thoughts on why it is that developers don’t implement changes that they know would be helpful. The problem is that although we know exactly what doesn’t work right and how it should be fixed, most of us will never say anything. We don’t say anything because there’s a very good chance the minute we do we will be marked as uncooperative, pessimistic, or simply detached from the business reality. (emphasis in original) He concludes with his call to action. If more of us say what we know in our hearts to be true, the rest won’t be able to ignore it anymore. Hopefully. And I agree with him. What he is talking about is an aspect of professional integrity. If you are a professional that a business relies on for good decision making, then you need to act the part and provide reasonable suggestions wherever they may do good. But there exists both a flip-side and a problem with Lidor’s points that are inherent in professional integrity that need to be explored. The Flip-side While you avoid negative attention by remaining silent about needed process change, you can gain positive attention by supporting whatever has caught the eye of your executives. This is a major contributing factor to “The Silver Bullet Syndrome”. At XanGo for example, we had one company convince management that they could cut development costs to 10% of their current level. Not 10% off, mind, that’s 10% of current levels. Despite all logic to the contrary (I mean, think about it–if a product could actually deliver cost savings of 90%, every company on the planet would be using it–they’d be incompetent not to), XanGo ended up spending millions of dollars and wasted a full year of development on this silver bullet. What was most fascinating to me were those who were willing to endorse this silver bullet. Those people were promoted to team leaders and project managers. Any who would oppose the new development project were either removed before its introduction or told (in at least one case explicitly) that negative feedback would put their jobs in jeopardy. In this way, technical people who should have known better ended up perpetrating the problem not by staying silent but by actively promoting bad solutions. They were rewarded for doing so. Here’s the thing: in most cases, there really is no price to pay for backing a losing silver-bullet solution. Oh, those in front of the board have negative exposure when it all blows up. Some of the higher management tiers are at risk as well. But for most of IT, there simply is no bill to pay for being in the pack. The Problem Here’s the thing: having Professional Integrity carries not-insignificant risks. Having integrity makes you predictable. Those with an agenda can plan their actions confident of what you will do. That means that if things get nasty, you end up following Marquess of Queensberry rules in a bar fight. Your compensation is in the hands of ignorant people. They don’t know what you know. Many of them may be smarter than you. Even if they aren’t, though, they are still the ones signing the paychecks. There is seldom a single, obvious best answer. Honest and rational people can arrive at differing conclusions based on the same information (because they weight aspects of the problem domain differently). This can easily split the message received by management and open the door for bad solutions. This also leaves you vulnerable to the manipulative because they can confuse the issue at need. Costs of bad solutions are often deferred while the costs of changing minds is immediate. Whether opposing a bad proposal or proposing a better process you are asking for change and that makes people uncomfortable. If you make people uncomfortable they will often hold you responsible for their discomfort. Sometimes having professional integrity means looking for a new job. Whether you are fired or are forced to resign, some positions are simply incompatible with maintaining professional integrity. These situations are rare, but they can happen. People aren’t rational. As a result businesses often make choices that seem irrational. Sometimes those choices are right even though they are irrational. Professional Integrity in such a situation is problematic at best. As I see it, Lidor’s call to arms makes sense but is unlikely without more support. The thing is that the problem is bigger than he has acknowledged. He is asking us to take actions that are overtly risky and he hasn’t given us enough of a reason to do so. I mean, the emperor may not have any clothes, but there’s no telling how he’ll react to you pointing that out. The Solution Domain “Because it’s the right thing to do” resonates with most of us, I think. But it is important to acknowledge that, at heart, Lidor’s plea is a moral and not a technical one. As such, it needs to be approached from a social, not a technical standpoint. The thing is, we are equipped to make this happen, but we need to explore it in the right context if we hope to make progress. Societies have a lot of experience coercing correct behavior. There are a lot of tools available, some better than others. My preferences tend to flow from my LDS background. As such, I’d advocate proselytizing correct procedures. We should explain as clearly as we can the benefits of acting correctly. While the payoffs of acting correctly are long-term, they exist nevertheless and are provable and measurable. Professional Integrity in IT has similar long-term value (above and beyond correct practices) to businesses and those who learn to identify and value it will gain the benefits it brings. A company that hires professionals with integrity will do better than companies that hire similarly talented people who lack that dedication. As long as we’re borrowing from religious tradition, lets hit up the Catholics and/or Jews (leaving no stereotype unturned) and apply some well-earned shame. I know who sold us out at XanGo. My reaction to them in the future can (and should) be informed by their actions there. I think we do ourselves a disservice if we know compromised IT professionals and we don’t tell them our opinion of their actions or make those opinions public. Repentance (more religious terminology, but applicable) should be possible, certainly–people can change and reformation should be encouraged. That said, it’s important that forgiveness be contingent on honest efforts towards reparation and change. On a more general level, ostracizing those who behave badly is one of society’s greatest tools to restrain bad behavior. The next time I review resumes to evaluate new hire candidates, I’ll have an eye out for those I know who, while they may be bright and talented developers, ended up endorsing those things that were bad for the business. More generally, I’ll try to craft questions that explore a candidate’s professional integrity in addition to their technical expertise. Reputations in IT should be more than merely technical. Professional Integrity is important not just to businesses but to the perception of IT as a whole. The IT crash in 2001 contained a lot of payback for abuses in IT during the build-up of the bubble. While we were on top many of us made unreasonable and expensive demands on companies (some personal, some technological) and those abuses weakened us as a group. Collective bargaining is one route that I’m sure will come up–a union or standards body that enforces “correctness”. Personally, I’m not a fan of that option. Bureaucracies are inefficient and tend to be in opposition to business interests. We do not need more antagonism from the executive suite. Unions also have a tendency to groupthink that I consider inimical to technology innovation. I’m sure there are others that I am overlooking. I’d love to hear further thoughts in the comments. Please leave ‘em if ya got ‘em. Don’t make me beg. I acknowledge that my thoughts here will be hard to implement. They require conscious short-term effort with no promise of reward. It’s certainly beyond the scope for a single individual to affect. Still, I believe it’s possible to make things better than they are right now and that the potential for reward is both real and achievable. Plus, Paladins are made for exactly this kind of idealistic fight for what’s right… Technorati tags: Integrity, Development, Professional, Honesty, Software Posted: Aug 28 2006, 11:23 PM by Jacob | with no comments Filed under: Development, Software, Honesty, Integrity, Professional […]