4 Weeks Incident

Hi,

What happens when software rots over decades?

More than I can list. The software has accumulated so many problems that they can’t be described compactly.

• The monitoring is incomplete
• The deployment is cumbersome
• The infrastructure is heterogeneous
• Libraries are outdated
• Knowledge has been lost
• Background and context for decisions are missing
• The code is tangled
• … and has high coupling

So far, this is not surprising. Maybe you have worked on such a project yourself.

But what happens when such a project is successful in the market?

Initially, it sounds great - and it is - but in the last four weeks, I experienced how this can drive you into a technical dead end. At full speed. And everyone notices.

What happened?

Four weeks ago, one of my clients noticed that sometimes report generation didn’t work. This didn’t always happen. Only some customers were affected.

After support looked into it, it was passed to the development team for investigation after a few days via the PO. But there wasn’t much to find out. When we logged in, everything worked.

So the team discussed one or two options - but it was dismissed as a “hiccup.” After all, the software is so old - such things happen. This is also a form of resignation 😉

But a few days later, the reports became more frequent. It became clear - this is not a normal “hiccup.”

The project reached a tipping point. Too many users were using the feature. And the feature had a hard limit.

But everything was missing to analyze this problem. First, the monitoring had to be expanded. That took many days. We discovered one problem after another - only to discover the next one after that. And on top of all this, the software can’t be deployed without downtime. So every patch had to be well thought out.

Don’t let it come to this!

Software that rots will eventually reach a tipping point. And then it will be painful. Especially for the business.

Rule the Backend,

~ Marcus