Fooled by Randomness

September 10, 2007 CollabNet Team

Is software development a science or engineering discipline in which we can strive for perfection? Are perfectly executed software plans possible in today’s market? I believe that software development, in general, is fraught with uncertainty and is therefore governed, at least in part, by random events. Assuming for an instant that this statement is true, then there are tremendous implications on how software projects are developed and managed. The Agile movement is largely based on this concept, yet it’s not a slam-dunk for many people.

In the pop-finance book “Fooled by Randomness“, the author N. N. Taleb makes a case that most, if not the whole, of the financial markets industry is governed by chance; randomness, not the skill of individual market traders, accounts for the failures and successes of the numerous funds and similar financial vehicles. Taleb argues that our population is generally uneducated and inherently unskilled at dealing with the statistical reality of situations fraught with randomness (like financial markets), and that certain biases (like the survivorship bias) skew our thinking to confuse “success” with “skill”. He argues that it is actually luck that determines success in financial markets, not the skill of individual traders. Taleb contrasts the financial markets industry (purely “luck” based) with other industries that rely less on luck and more on skills; he uses the example of dentistry as a profession that relies largely on the skill of individual dentists as opposed to how lucky they are in determining success.

Where is software development on this luck-skill spectrum? I believe software development lies somewhere in between financial markets and dentistry on the spectrum of luck vs. skill based industries. Certainly, the skill of the people involved has an impact on the outcome of software development projects. However, I do believe that software development is fraught with uncertainty and therefore relies to some extent on good luck. Here are just a few areas that I’ve observed to cause uncertainty in software development projects:

  • Software is not a physical science. Unlike bridge building, there are no physical laws that underpin software development. As a result defects are common and frequent. And further, we build systems on top of platforms that were themselves built on shaky foundations.
  • Changing product goals due to business climate. Software is typically built in response to business needs, not the other way around!
  • Interpersonal interactions. There are many people and many personalities involved and the interaction of complex systems (like people) can rarely be predicted.
  • Personnel changes. People come and go, and software knowledge workers are not “fungible resources” (like it or not!).
  • Software is hard to visualize until there is some software to sample. “I’ll Know It When I See It” is a common refrain.

This list could go on for a while, but hopefully you get the general idea. For a more complete list, see Craig Larman’s excellent book “Agile and Iterative Development: A Manager’s Guide.”

(Important Aside) Note in the above paragraph that I used the word “uncertainty” and not “risk”, as there is a big difference: risks can be mitigated while uncertainty cannot regardless of the measures we take. I’ve heard many software professionals talk about risk mitigation as a driving force in software methods. Unfortunately, risks are only one side of the coin, the other is even nastier: complete uncertainty that cannot be resolved regardless of the time spent (think BDUF) trying to mitigate.

So it appears that software development is more a blend of art and science rather than a strict science that can be calculated with mathematical certainty. While this may be a hard pill to swallow for some, it certainly makes sense that Agile methods like Scrum and XP are better equipped to juggle the complexity of software development than traditional development methodologies. Traditional methodologies take a formulaic approach to development, featuring libraries of cookie-cutter prescriptions to account for all or most possible scenarios. Of course this won’t work very well with a nearly limitless set of possibilities. It is best to set high level goals and work in small steps toward those goals, making adjustments for the reality of the situation along the way.

If this is true, then there must be a collective realization that software development projects cannot be controlled like other “engineering” projects. We’re not building a bridge, or even shooting a rocket into space; we’re doing something harder. The iron triangle variables of cost, schedule, and requirements cannot be fixed in advanced with any expectation of project success. But the Agile folks have been saying this for a while yet many people out there still hang on to the belief that software can be “calculated” by applying more rigor to methodology. To those people, I suggest reading Taleb’s book even if you are not interested in financial markets because it highlights a common flaw in our collective psychology, the fear of the uncertain to the point of denial. Our instincts (in general) are to cling to anything that promises predictability and repeatability. But if we are to succeed in disciplines that are affected by randomness to a large degree, software development among those, we must try to fight this instinct and open our minds to dealing logically with uncertainty.

Previous Article
Subversion merge made easy

CollabNet released a preview of its GUI merge client for Subversion 1.5. You can find the software at http:...

Next Article
Subversion Merge Tracking – The Basics
Subversion Merge Tracking – The Basics

Subversion 1.5 will support Merge Tracking. This blog post provides a high level overview of what Merge Tra...