The Art of Writing Software

Cracking down on technical debt

Tags [ architectural debt, configuration debt, technical debt, transition debt ]

“Simplicity is the ultimate sophistication.” –Leonardo da Vinci.

“Everything should be made as simple as possible, but no simpler.” –Albert Einstein

“A designer knows he has achieved perfection not when there is nothing left to add, but when there is nothing left to take away.” –Antoine de Saint-Exupery

I’ve written before about the notion of technical debt. In this post, I want to discuss a few specific sources of technical debt that are easy to accrue, particularly in an agile, iteration-based setting.

Incomplete technology transitions

These can arise when a technical decision gets made to transition from one technology/architecture/design to another, and the transition happens incrementally. What can end up happening is that an agile team, say one operating under the Scrum framework, does not complete its incremental transition during the current sprint. Now, although the code is in a working state, there is a good chunk of technical debt arising from having code operating under two separate systems. This transition debt is problematic for a few reasons:

First, this can complicate debugging efforts – when there is a problem with the system, someone has to determine under which scheme the code in question was written. Typically this can mean looking in two different source code hierarchies, or looking through two separate sets of configuration. The system is, as a result, more complicated than it needs to be.

Secondly, this can be an attractor for additional debt; if the old system is still around, and a developer is more familiar with the old system than the new, there is a very strong temptation to make changes/additions in the old system. This work simply adds to the outstanding transition work, and despite the developer’s familiarity, is likely to be implemented in a more difficult or less efficient way (assuming, of course, there were valid technical reasons for making the transition in the first place).

Finally, this can cause extra work to happen during feature development that touches/interacts with the subsystem in transition, because either the cooperating subsystems have to special case two different interaction styles, or an adaptation layer has to be built to handle both subsystems and abstract their existence away from clients’ concerns. Either way, you are writing more code that you would have if you had completed the transition and you just had one implementation of the subsystem.

Teams working on an iteration-based methodology need to do several things to avoid the pitfalls from transition debt:

  1. when a technical decision for a transition has been made, it must be communicated clearly to the whole development team, including the reasons for the transition. This can help prevent the unintentional accrual of additional transition debt.

  2. plan for more refactoring time when signing up for work, to leave time to complete transitions before an iteration ends.

  3. communicate the existence of the transition debt to the Product Owner at the review, so that completing the transition can be scheduled as a backlog item. Furthermore, stress the priority of this carryover work to ensure that the transition debt exists for the shortest amount of time possible.

Obsolete/extraneous configuration

We’ll call this type of technical debt configuration debt. There are a couple of sources of this type of debt:

  1. transitional runtime configuration that still exists after the transition. For example, when a data partner was making an id space transition to extend the length of their ids, we had a flag to govern whether to use old or new ids with that partner, so that we could decouple our code releases from the partner’s transitions. Over a year later, the flag still exists, but it is always set to use “new” ids, so there is certainly unneeded code to handle this.

  2. exposing properties that would only change with a code drop as runtime configuration. In this case, the values of the properties would really only change if we rolled a new code release, so they could just as easily be compile-time constants that would not require the scaffolding to make them runtime properties, no matter how simple that scaffolding might be.

Unnecessary code hurts you in several ways:

The easiest way to prevent the accrual of configuration debt is to review any new runtime configuration parameters at the end of each sprint (which you probably have to do anyway so your operations folk know how to properly configure the new system). Then, where possible:

Obsolete/insufficient architecture and design

Architectural debt is probably the most nefarious, because this is debt that doesn’t actually get created when the code is first written. Instead, this is usually caused by external factors such as:

Basically, as soon as you realize you need to change your architecture, you have magically “created” technical debt out of all the code that depended on the first architecture. In reality, this debt is probably unavoidable, and what you’ve really done is convert your inability to perfectly predict the future into a set of work that incorporates new knowledge about the problem domain.

This can also be hard to identify by scrutinizing the code, but there are some external symptoms of it:

So when you have some or (gulp) all of those symptoms, you probably have architectural debt lurking in your system. Once you have identified it and have a new target architecture, a lot of this will get converted into transitional debt while you are making the changes.

Technical debt vs. technical investment

I want to be careful here to distinguish between two sorts of non-functional requirements that might show up on a “tech backlog”:

Technical investments can probably be put off while you have existing technical debt, although it can sometimes be hard to distinguish between the two. Clear technical debt should probably be prioritized at the top of a product backlog, unless there are really high ROI items that might trump it. In general, getting rid of technical debt will increase the ROI of everything else on the backlog, simply by decreasing the “I” part. It can also make estimation more accurate by reducing the complexity of the system to which new functionality will be added.

Whose responsibility is technical debt?

Generally, as the folks with the technical ability to recognize it, it is the development team’s responsibility to try to avoid accruing technical debt while producing product. Failing that, it is their responsibility to recognize/document existing debt and to advocate for its removal. However, note that there are often symptoms of technical debt, such as those I’ve listed above for architectural debt, that can be recognized by non-technical folks too.

On the flip side, business folks / product owners need to be able to trade off short term wins that accrue technical debt vs. taking longer to produce a product with less debt. Communication with the tech team is of vital importance here; undoubtedly there will be times when a short-term win will be important (especially with a first-to-market situation), but it needs to be accompanied by a plan to eliminate the accrued debt. i.e. Treat your technical debt like credit card debt that should be paid down ASAP, and not as a long-term mortgage.

The interest on your technical debt is probably not tax-deductible.