Technical Debt – The High Cost of Future Change

December 21, 2011 Patrick Wolf

But little Mouse, you are not alone,

In proving foresight may be vain:

The best laid schemes of mice and men

Go often askew,

And leave us nothing but grief and pain,

For promised joy!

Robert Burns wrote this verse of “To a Mouse” in 1785 and his words are just as true today. Foresight is not a perfect tool so we aren’t capable of creating a perfect plan for the future. Even if we spend months researching there will always be known-knowns, known-unknowns, and unknown-unknowns to account for in our planning.

Things that we know to be true today can always change in the future. Anyone who has finished (or, I should say, started) a software release of any size knows that requirements are not constant. Business needs to constantly adapt to stay current with the marketplace. Business software follows the requirements of the business and must change just as rapidly. Software that forces the business to adapt to it rather than adapting to the business finds its way to shelf quickly. 

In addition to ever changing requirements we have to consider the ephemeral nature of technology itself. Will our state-of-the-art technology still be cutting edge by the time we finish? Is the technology stack too old or too young? Too old and it might perform poorly or require expensive refactoring later. Too young and it might be unstable and poorly supported, especially in a partner ecosystem.

Those are just the things we think we know. Most of us who survive high school realize at some point that we don’t know everything. In fact, there is quite a bit that we don’t know – I feel dumber every year. However, even in this great pool of ignorance there are things we know we don’t know. The rest are things we never even considered. On every project I have been part of I have always gone in knowing all the questions I didn’t have answers to yet. Likewise, on every project I have worked on we have come across problems that we never even considered when we started.

As an example, a few years ago I bought a wooden sailboat to rebuild. I researched and I schemed for months on ways to save my nice teak deck. Finally, I decided that I couldn’t save the deck because the plywood substrate was just too rotten; I would have to rebuild the deck.

Deck PhotoI knew my budget, I knew my meager carpentry skills, and I knew what I wanted as a finished product. I thought I had an achievable plan. At the same time, I knew that I didn’t fully know the state of the frame under the deck and some of it might have to be rebuilt. What I never considered and didn’t know until I removed the deck was that my steering quadrant was rusted and needed to be completely abraded, rustproofed, and repainted. I also didn’t know that the plywood substrate under my deck extended under my bulwarks (the section of hull that rises above the deck) compounding the rot problem by an order of magnitude. 

In this case, my unknown unknowns (the things I didn’t know and never thought to ask) put the project well beyond my time projections. While it’s true that more research and planning up-front might have revealed these problems before I started, it would not have changed the work needed to get the boat ship-shape again and I would have started even later. At this point I was faced with the choice of cutting corners – possibly compromising the future safety of the boat – to get something done before rainy season started or stick with my plan and finish much later than I wanted. I was facing Technical Debt.

While Technical Debt can occur in a boat-rebuilding folly such as mine it is more commonly associated with software development. When we begin a new software project we are typically given a list of feature requirements and a release time frame. This intersection of features and time is usually based on an ideal development project. As the known-knowns change, the known-unknowns crystallize, and the unknown-unknowns manifest the ability of the team to meet the original goals is compromised.

In the case of my boat the debt resulted in a long delay in using it. If I had cut corners to finish the work in one season the cost of future repairs would have been much greater, assuming future changes were even possible. Software development teams don’t have the luxury of delaying a release indefinitely; something has to be shipped. When faced with missed deadlines most software teams go into Technical Debt. The release sometimes takes precedence over good development practices: features are dropped, tests are skipped, and extensibility is sacrificed. The future cost of paying back this Technical Debt through refactoring, creating new features, or updating the technology stack is often overlooked.

Tech Debt Webinar Recording

Adopting Agile development practices is a great way to combat this mounting Technical Debt. Rather than go into the details I highly recommend watching the webinar hosted by Certified Scrum Trainer Michael James. He chronicles several examples of Technical Debt and discusses how best to use Agile development practices, specifically Scrum, to lessen the occurrence of future debt and incrementally pay back accrued debt.

Another factor in reducing Technical Debt is to make sure that the required tools and the correct tools are always available to everyone on the team. “Mise en place” is a French phrase used in culinary schools that means “everything in place.” Chefs are drilled in the practice of preparing their stations before beginning work so they can quickly deliver high-quality products on a consistent basis.

Agile development teams should approach projects with this same discipline of “mise en place.”  Developers don’t need just an IDE; they also need access to collaborative tools such as version control, issue tracking, file storage, and agile management. Having these tools available and ready-to-go when the project starts enables agility. Rather than devoting resources to installing, maintaining, and supporting all of these tools separately more and more teams are turning to the Cloud to properly equip their development team with all of the tools they need.

This is why Agile software development and the Cloud make such a powerful combination. The methodology is designed to make the team as efficient as possible and the platform is designed to deliver the requisite tools in a highly available, scalable and secure format. The two together can significantly reduce Technical Debt and lower the overall cost of delivering high-quality products in a timely fashion.

Previous Article
Subversion Edge 2.2 Released
Subversion Edge 2.2 Released

Subversion Edge 2.2 was released this week and brings some exciting new features that our users have been a...

Next Article
Scripting TeamForge Connector Server (CCF 2.0) REST API for fun and profit
Scripting TeamForge Connector Server (CCF 2.0) REST API for fun and profit

One major feedback we received on CCF 1.x was that our customers really liked its bidirectional artifact sy...