10 Facts About Technical Debt Every PO should know

Click to download the slides of this post

Technical debt is actually something we want.

At least early in a project.

I know that’s counterintuitive - let me explain.

In the later stages it can be deadly for a project. It can reach a tipping point where only a full rewrite is possible.

This post contains 10 things I believe are true from my humble 20 years of software development.

#1 It’s not reasonable to try to prevent technical debt altogether

Many founders do this:

• 🤔 Build something you think people need
• 👩🏻‍💻 Waste a lot of time on the implementation
• 😅 See that customers don’t care
• 😮 Be stuck with a rigid piece of software that takes a long time to refactor

My point is this: In this case (before Product-Market-Fit) you want technical debt as it’s going to be less than the technical debt you’ll build up with the big solution.

Why?

Because you want to delay a decision:

The decision of “Is this feature actually moving the needle for us?”

Yes, by all means: build up debt to find out.

Make some money. Stepsize got a point:

Shipping a mere two months later will cost our business 25% of the revenue we could’ve made over this selected period.

And then refactor to get it right.

But what to do when the something works and then the product should scale? How do you deal with technical debt then?

Go ahead and read up until point number #10 “The tipping point”, where I explain what need to get done in this case.

#2 Most of the time devs don’t produce it knowingly

Well, very experienced developers might catch themselves producing it. Some of the time.

👨‍💻 Inexperienced developers:

I’ll skip the tests here; we can manually test this later anyway.

If your developers have this kind of mindset, you need to be alarmed. Skipping the tests is rarely something that saves you time. Any small change, even if you are pre Product-Market-Fit will take longer to develop. And then there’s TDD. In many cases this makes things faster, event if you are doing something quick and dirty.

🧙🏽‍♀️ Experienced developers:

When senior devs are working a project they also pile up technical debt; but that’s happening with tests, with specs, and with good management.

How come?

Because we don’t know how flexible a system should be later on.

In order to be quick, they build structures less flexible, but faster.

This is how I imagine technical debt in my head:

Basically it’s this: you can build a correct foundation, only if you can see where you need to go. The red line is not visible when you start out.

#3 It’s always there and always needs to be dealt with

The previous graph already explained it a bit.

If you still have doubt - how about looking at the engineering wizards from silicon valley?

Even they have to deal with it all the time.

Stripe engineers spend 33% of their time dealing with technical debt.

Read this nice PDF from stripe here.

I can confirm.

In client projects we sometimes keep a separate swim lane in Jira for technical debt. We tried so many things, which I’ll work through in another post.

Some ideas I’ve seen:

• Developers spend every Friday to refactor (people loved it and it was definitely enough time /sarcasm)
• We initiated “Ranger” teams where 2 people only refactored and tried to keep the software running
• The “DVD - Daily Duty” person who spend every day refactoring.

Some of those worked better than others. The takeaway here is this: once a software is in use and you cannot just scrap it and make it new: you’ll have to deal with technical debt.

Some general principles every developer knows:

• For obvious cutting corners: write a TODO in the code.
• For bigger things: create a dedicated ticket/task labeled with refactoring.
• No developer should feel guilty or get blamed for cutting corners, if it’s planned.
• Every code that brings along technical debt should be discussed

In general: strong code reviews are important and should be done before any non-trivial merge. But that’s not all: Refactoring should be part of the SDLC (Software Development Life Cycle). Point #8 describes this a bit more.

For further reading, Martin Fowler has a great book on the topic refactoring.

#4 One of the best ways to minimize debt: less code

Of course, nobody is going to tell you this: Let’s just write less code.

But it’s the biggest predictor of how fast you can refactor a code base (obviously). Write less lines, delete less lines. As simple as that. Everyone knows Occam’s Razor

The simplest solution is almost always the best.

This is how it looks. Picture taken from radicalsimpli.city - which is a good read on the topic.

Now, the question of course is, how to do that.

I write most of these posts with Ruby on Rails. In case you are somewhat new to it, it’s an opinionated framework that promotes the following:

convention over configuration.

While it’s possible to do things many different ways, there’s usually one way that’s the simplest. The framework then goes with the built-in convention.

In effect, you write less code. Less code means less bugs. Less technical debt.

It doesn’t stop at less code. Every additional dependency, library, and framework you add to the system introduces complexity.

If you don’t believe me, read this post. It shows how badly frontend frameworks fail to deliver on their promises.

There are cases where libraries help reduce complexity. Whenever I use a new package and I can find someone who has worked with it, I ask them about the drawbacks and how it worked out in the long term. Often, additional libraries sound fun at first but make you suffer later.

Apparently there’s a term for it: Tooleritis, where for every solution a new tool is used, or a new service is plugged into. More dependencies = technical debt.

#5 Incentives drive the outcome

If you are a developer you might get mad at me now.

Fact is: developers are like small children. Distracted by shiny things they can play with.

It’s worse with hourly-paid freelancers. I’m not kidding - that’s what I’ve heard from a dev I met in Eastern Europe:

I’m always joining interesting projects with a new technology. I try to learn as much as I can. Then I switch jobs.

When reviewing this post, I got asked the question how to spot if a new hire has this kind of mindset. Quick answer: ask about their goals, and look at their employment history. The topic needs more explanation, which I’ll do in another post (subscribe to my emails to get notified).

Of course, that’s an extreme example. But you cannot tell me that this is not something you’d like to do (if you are a developer, manager, founder).

How this can harm your project?

Well, projects follow the Parkinson’s law: a task expands to its boundaries. And I’ll add: the complexity expands for the task’s allocated timeframe.

Practical example I just made up:

• You give a developer the task of implementing an importer for a CSV file.
• Simplest implementation: 2 hours
• But as there’s no deadline pressure they might use a new library for generic CSV parsing. Then read everything about it. Then use it, play around with it.
• 16 hours are wasted, which the freelancers bills you for.

–> Now you have quite a bit technical debt (a new dependency), and maybe you don’t even need the importer to begin with. Or the importer should be able to read zip files, which is not possible. Reworking it now takes longer with the library.

If you don’t believe me, just read up on the cobra effect and see how bad incentives often drive perverse outcomes.

#6 Team’s Morale can diminish if you don’t address it

Take this one with a grain of salt. Technical debt usually doesn’t scare good developers.

However, devs get angry then there’s no time to deal with it.

It’s like having a roommate: usually it’s fine for you to pile up dirt in your house. But once you have a roommate, you get annoyed by their dirty dishes real fast.

The problem is this: developer experience degrades with increased technical debt. And once things are not taken care of, things fall apart faster and faster, see the broken window theory

On the flip side: developers can have a lot of fun tackling technical debt.

It can feel great. It can feel like ownership over a problem.

Managers should fight for their team to get the time to refactor things constantly without needing to ask for a special budget. Some refactoring should be built into every iteration.

#7 Agile software development won’t be the remedy

Talking about iterations: many software teams are agile, nowadays.

Does that mean technical debt is less? No.

In fact it might induce more technical debt as less planning is done upfront. And in hectic sprints with ambitious goals there can be less time to remove technical debt.

Agile only helps with technical debt - if the agile team deals with technical debt.

In case you know exactly what you want to build, consider going waterfall. It’s still a good model when planning big software projects.

#8 Testing + Refactoring should better be in the DNA of your team

I had a call with a fellow consultant I worked with, Andy Wenk. He once gave a talk about the topic, and one thing made most sense to me:

Testing, Refactoring, Monitoring belong into the DNA of the team

Steve McConnell put it well in his book: “Professional Software Development”

We call this the mid-way between full upfront design and planning and having to constantly rewrite entire software implementations of systems. That’s where the good modular design of components and containers comes into play.

#9 Clients need education

In the chat with Andreas I noticed that often clients are not willing to “pay extra” and don’t see the value in reworking an already written subsystem.

As he told me: managing the technical debt requires strong leadership towards the client.

Basically, what needs to happen is this:

0. Plan refactoring into the budget
1. Document every time you pile up technical debt
2. Work through it not only in one season (like the dreaded “Quarter of refactoring” but better constantly)

#10 There’s a tipping point

Yes, there are projects where too much technical debt slows down development so much that no real progress is done.

The longer we put off maintenance work, the less smooth and effortless feature development will be.
– Chelsea Troy on stackoverflow blog

That’s usually the point when management adds new meetings or more managers.

That’s also usually the point when developers leave, and only mediocre ones are left to handle the mess.

Good managers continuously test how close the project is to the tipping point. The best way to do this is to monitor ticket throughput and developer happiness. I’ll write more on this in another post.

In the startup world, this point hopefully comes early, right after Product-Market-Fit:

Jordan from Millionaire Millennial points out that many startups needed a full rewrite of their codebase once they validated the Product-Market-Fit. Timestamped for you here:

https://youtu.be/Iigu8GBq2E8?t=237

His point is: Stop overengineering and get the MVP out. [Even if it has technical debt]

When the tipping point is reached, there are only 2 ways to go about it: a full rewrite. Continuous adaptation might work, but is often slower and churns more developers. This is a complex topic which I don’t have too many data points on. If you know someone who I should interview on this topic, let me know!

Conclusion

What points did I miss?

I’m writing those as I go along and learn from people. If you got input here, I’d be happy if you write me!