I once again had to step up and show my superior management skills (or SMS – pronounced sims by my VP) by looking into an incident that happened late yesterday/early this morning. Once again, our manager is not here – either sick or taking care of his sick wife – leaving me and the other Roger to fill in. I took the initiative on this one, since I hate having loose ends.
As Provisioning Engineers, we either turn-up new Internet connections or disconnect old Internet connections to the ATT cloud. That’s 80% of my job now – creating and deleting interfaces, ordering new circuits and directing on-site techs on how to ensure the client is up. I probably spend more time on the phone with the client than doing anything else. That’s our jobs, really.
We can cause issues, but one particular issue is considered bad bad bad. The Disconnect in Error, or DIE for short. That’s when we accidentally disconnect a client from the Internet without their official approval or a record to do so. This is why we double and triple check orders and the interfaces. If they are up, ping-able and passing traffic, we ask the Service Consultant or Specialist to confirm with the client. We also push it out to a Subject Matter Expert (SME) to make sure we’re doing it right.
Well, one of the PE’s had a DIE. Luckily, he kept all his emails and messenger files. It clearly shows the Service Consultant and Specialist at fault, because the SME and the PE were both led to believe it was okay to disconnect the client. So yesterday, the PE disconnected them. This morning, another PE had to turn the client back up, because the client wasn’t supposed to be down.
With the notes the PE took, I was able to formulate a proper response to upper management with proof that it wasn’t the PE’s fault. So the PE keeps his job or doesn’t get written up. Without those emails and messenger notes, I would have been dead in the water and the PE would probably be facing some harsh punishment.
So I say to all you programmers and engineers – take copious notes when you do something that impacts the client and you have a bad feeling about it. Because then we managers can step up and pull you out of the fire. I cannot stress how important it is to make that a part of your routine everyday.