The Never Ending Bug

Have you ever been faced with a bug that feels like it’ll never end? As one door closed you find four more opened up in their place, and the complexity multiplies? I believe it’s safe to say we’ve all been there, and possibly years later the after-effects of the bug still remain.

To fix or not to fix

Often time in situations when working, you’ll need to close the bug, but if other pieces of software rely on this working exactly as it is, you’re now risking breaking entire applications as a result of poor design choices. So, the management tells you not to fix it, instead patch around it. They don’t want to invest the time properly fixing it as it’ll cost more money. Logic and reason doesn’t always work, so what can we do?

Let’s put up a firewall!

If you haven’t already, putting up a firewall is important — but in this situation, the application is public and needs to work without being knocked down by false-positives, no rule is 100% bulletproof. So what next?

How about some ACL?

We’ve been told we can’t fix the bug, and implementing Access Control Levels would require updating other applications, so this is a non-starter.

IP Whitelist?

This seems like a temporary viable option, of course, until the dynamic IP of the customer changes — when working in SaaS environments, you’re not always assured the third-party won’t change their origin IPs. It’s happened several times to me where other businesses will float applications server to server, even cloud to cloud, causing IP whitelists to fail.

So, what’s the solution?

We’ve analyzed three options to patch around the issue, but these are all non-starters. With each passing rule, if we decided to implement web-application firewall rules, we’d still be left with a moving target – now if we change the spec of the API, we need to roll out WAF changes to patch around it. If we implement ACLs, we need to fix the application to validate the ACLs. Failing this, we implement IP whitelists, and then mandate our customers give us 48 hours notice of an upcoming change, which is not realistic given the fact most systems are automated and will do these changes autonomously in the background.

Convincing the boss

If you’re in this situation, it’s best to try to convince your boss to implement the proper fix, regardless of the excess time it’ll take. You don’t want to be the next big data breach. As a former manager, one consideration I always took into account was the technical debt down the line. The key, in my opinion, to successfully convincing your boss to authorize a proper fix is showing that the time to fix now will result in less technical debt down the line. The bottom line is you either pay it once now, or twice down the road.