The Art of Firefighting - Part 1

Firefighting is one of the most common experiences that everyone involved with software development goes through. Irrespective of your domain, country, organization, or experience level you are at, if you are involved with writing software, you have to go through this trial by fire.

Scary as it sounds, these are also the moments of highest achievements that bring out the best in you and form part of your most cherished work related memories. This is the stuff that your brags at friends evening out and anecdotes narrated at job interviews are made of.

While everyone shares the burden, tremendous onus lies on the manager to turn these testing times into pleasant memories. A few crucial decisions made during these stressful times are what differentiate the cherished memories from bad experiences that taught everyone one more way of not doing things.

So what is it that takes to lead a successful firefight?

First and foremost, it takes calmness and composure. Easier said than done, I am afraid. It is all too easy to get hyper in the thick of things and act on impulse. However, it takes a lot of self control and effort to stay calm and focused. This is extremely important in order to retain the capability of making rational decisions when it matters the most.

If you are in a fire fighting mode, chances are you are going to run into one problem after another, some foreseen and some unforeseen. Your composure would let you select the right response to foreseen problems and devise smart workarounds for unforeseen ones.

Think of it as crossing a forest. If you are going to start running hard and fast, chances are that it is going to become a very very harsh ordeal with unpleasant ending. However, if you have the composure to stop and listen to jungle sounds, invest your time in climbing a tree to get some sense of surroundings, take into account time of day and need for shelter, chances are you would cross the jungle and live to boast of it during your evenings out with friends ;-)

Here is a more relevant example. You are end of the integration testing phase (which is a hotbed for many a crisis) and your backend API team has just pulled an all night miracle fix enabling rest of the teams to continue with the IT. Its early in the morning and they are winding up, hoping to get some well deserved sleep. What is wrong with this scenario?

Ans: Late in the IT phase, you would be proceeding with IT without a core team in office. Imagine what happens when a bug pops up (as it invariably does) and initial findings point a finger of suspicion towards the backend API implementation. Imagine how much it can slow down other teams when API team is missing from discussions and clarifications get delayed. Worse, how much combined productive and precious time is lost. While it might feel right that entire team is working hard to keep the deadlines and they are all motivated and pumped up about the delivery, these feelings will quickly evaporate when the pumped up team is blamed for delays and lack of support.

If an all night was absolutely necessary, some backend API team members should have been sent home last evening so that they can carry on IT with the rest of the team the next day. Some form of hand off also needs to be planned so that the day time fighters know what the night warriors did in the battlefield. These small foresights on the part of the manager are crucial to success of the delivery. However, it would take some composure to not give in to impulsive action of having everyone working to solve the issue asap.

Climb a tree, get a sense of what may lie ahead and you would survive to brag.

PS: If you find a relevant Dilbert for this article, please let me know.