Hastily urgency is hardly ever right

Oh yes, I remember why you should never directly translate sayings from one language to another. The non-Dutch speakers will have to take my word for it that “Haastige spoed is zelden goed” has a lot better ring to it. The meaning of the saying is fairly obvious of course “Don’t rush things, they usually come back to haunt you.”
We all know this and it’s even more true in IT, where small mistakes can have dire consequences. But we too often fall for it because with today’s complex systems it is hard to oversee what can go wrong. And it is even harder to estimate what it will cost you if things do go wrong. I would like to share a case with you in which I will explain what went wrong and try to give an overview at what it has cost us and our client.
To deliver one of the features our client had requested I needed to run a very small servlet on a different URL than the one our normal application was running on. There was no other way around it. The issue was that it was a lot of overhead to create a new module in Subversion, set up a maven project, deploy and maintain a seperate application for just 15 lines of Java code and 6 lines of JSP.
So the “solution” I came up with was to proxy that one URL and redirect it to a servlet in our normal application domain. This worked really well. The new functionality worked and everything seemed to be just fine. It was not until a few days later during a thorough testing session a day or so before the deadline that we realized that an obscure bit of functionality that we never even touched in the past 4 weeks stopped working. It had us completely baffled for a day until I finally realized what the problem was.
For the browser the external URL was not part of the application, so it did not send its session cookies with the request. The servlet did reside in the application domain however and thus concluded no session was available and promptly started a new one, removing all state that was present in the previous session. Thus every bit of functionality that relied on the session failed.
A case can of course be made that obviously our automated testing was not adequate enough. That is a valid observation, but this is an inherited, badly written JSF application that is very hard to test automatically and there was no budget to do full testing up-front. A lousy excuse, I know.
So that was the case of a shortcut that I thought was feasible, but in the end proved wrong. I tried to save maybe half a day of development time and maybe another day or so in deployment and documentation time in the long run. The end result is that it has cost more than two days extra in wasted development and debugging. That is more than 2000 euro for our client, 2 days less development time for other features, a loss of reputation for my team and my company and this was amongst a few other issues that made sure we couldn’t make our window to go into production early and this had to be postponed for another week.
This all has reminded me that it is always important to do proper risk assessment when taking these shortcuts. Something I failed to do when making the decision because I was under pressure to get things done quickly. It is a lesson well-learned.. once again. Let’s hope this post reminds people next time they are a position like that.. I know I won’t forget for at least a little while 🙂