A Bad Week: a study in journalling

Sunday was a good day.  I ran through Queens to Flushing Meadows Park and back, farther than I had ever run before.  Towards the end, I couldn't remember ever having walked; running was my natural state.

On Monday, my shift started at 7:30am, a half-hour later than usual; very relaxed.  A few minor aggravations at work, people not following through on what they were supposed to.  I had to tell a colleague that her database was badly corrupted and would need to be rebuilt from scratch.  She was inconvenienced but gracious, no doubt cashing in some goodwill chips I had earned in weeks previous.

Tuesday morning at 2:40am I'm awakened by a member of the night shift, not an email but a phone  call, requiring me to fake awakeness in my voice while quickly taking the conversation out of the hearing of my sleeping wife.  I had apparently missed a 12:40am email on this same issue, so before I even know what the issue is, I'm already an asshole.  No client reports are running on a production server; well the processes are running, just nothing is happening.  Turns out that the blocking process is an administrative script written by another DBA, and it's been blocking everything since 7pm.  We've essentially lost an entire night.  What's worse is that the process can't be killed, thus unclogging the server and allowing things to proceed.  I have to bounce the bastard, terminating everything and requiring night shift to start everything over, delaying the morning startup processes, which are due to start in about 10 minutes.  I hate getting paged when I don't get to look like a hero.  All I could do was apply the Windows XP solution, and restart.

So it's now 3:30am, and I'm due to be in to work at 7am to cover for my manager, the head DBA, who can't work early today for some reason.  No point going back to sleep; just tell the cat to brew some coffee and finish watching Arthur on Netflix Instant.  Try not to think about how tired you'll be later on.

Tuesday morning I find out that our hardware people are going to have to change the hostname of a pair of database servers because, wouldn't ya know, they just found out another pair has those names.  This will completely cock up the hour's worth of monitoring environment installation I did last week.  Ah fuck it, whatever.

I attend a meeting between developers taking orders from marketing folk and support people trying to educate the developers on how the software works.  The dev people are ignorant about the workings of the product that they maintain and the support people are confrontational and beligerant.  I don't know what anyone is talking about, so I just feel quiet and stupid.  My manager forgets until I'm just about to leave that I need to work the 7am shift Wednesday too.  This is bad news because he and I have scheduled a lengthy upgrade for Wednesday late afternoon.  I tell my wife my Wednesday shift will be 7 till question mark.

I run and do laundry on Tuesday night, falling asleep in my computer chair watching The Thing.  I should have gone to bed at 7pm.  

I wake up at 6:15am Wednesday, about 2 hours late.  From my Queens bed to lower Manhattan in 45 minutes?  Never been done.  Till now.  I punch in at 7:01am.  The perfect crime, except I haven't shaved or showered and I know I must work late tonight.  

The day is peppered with colleagues ignoring my emails then asking where they are, not apologizing.  I organize and chair a technical meeting of the three company DBAs on issues regarding database consistency checks and server standardization issues caused by poor DBA disclipline.  Throughout, my colleagues and I do not communicate clearly or efficiently, each choosing to understand one the part of the problem they can expound on, until I bring up graphs and charts to focus everyone.  I also chair another meeting that includes several team and department heads regarding the database disaster recovery test on Saturday.  We go over the proposed timeline that I sent out on Tuesday morning (and which I have not received any comments on), which suddenly everyone sees as full of holes and inconsistencies.  They are boisterous, bordering on joy, with every error they uncover.  We are friends and we acknowledge our ribbing is purely functional in nature, but this is still shit they could have emailed me yesterday instead of embarassing me in public now.

My wife calls in the afternoon to invite me to a movie with her after work, and I must refuse due to the impending upgrade and my requirement to be on call.  This is hugely devastating because my wife has essentially never invited me to a movie, whilst I invite her approximately every six hours, usually being rejected.  I would have loved to have gone, and I owed it to her.

The upgrade is to install a monitoring environment on a database server, and one side-goal is to walk the newest DBA through the process; this begins at 4pm.  Of course one bizarre problem after another is encountered: environment variables are not being set correctly by the shell, remote execution is denied from the standby to primary but not the other way, the servers and log files are not named according to department naming standards.  I finish at 5:30pm.  The DBA I was supposed to be instructing left at 5.  My manager has left long ago.

I finally get a chance to catch up on sleep on Wednesday night.  To this I sacrifice weight training and Max Payne newly arrived from Netflix.

I begin Max Payne on Thursday morning before work, and am largely disappointed.  Thursday is mainly okay; only one semi-frustrating meeting and one maddening interaction with my fellow DBAs.  They should learn to answer the questions in the email, rather than answering questions no one asked.  Thursday night I am bugged by my paranoid manager suddenly worrying about a project rolling out tonight, despite the fact that we discussed it in detail during the day.  I am forced to log in from home and recheck the specs to reassure him.

There is an unrelated problem at 9:08pm on Thursday night.  I was in an...intimate moment so I am unable to respond to the inital mail until 9:24.  However, my manager who in his paranoia has apparently been watching his BlackBerry like a hawk, attempts to call me at 9:11, failing that, calls the other colleague, who engages the problem at 9:16, despite me being the one on call.  This infuriates me on several levels: 1) why is my manager watching over my shoulder?  That makes me feel like shit, like he doesn't trust my competence. 2) why didn't my manager just fix the problem?  If he is unavailable to address problems, why is he answering email? 3) Since the problem is not a serious one, I am supposed to have around 30 minutes to respond.  4) If this is the only after-hours call tonight, my other colleague gets the overtime pay. 5) It turns out there is another support call tonight.  Now, our company awards a flat $100 for every night in which you have to answer a page.  So by my manager roping in the other DBA, has unnecessarily doubled our department's stipend expenses for tonight. 6) The criticism implied by my manager's action of calling my colleage after he could not reach me on a low-priority issue has made me feel guilty about sex.

On Friday morning I finish Max Payne and I finish being disappointed.  It was okay, atmospheric and unpredictable, but not a fine film.  One strange point is that it snows throughout the movie, and when I step outside on Friday morning the weather is identical, hard flurries, gray sky.  First day of spring?  I also confirm that my tax refund has been deposited.  That's a nice feeling.

I spend all of Friday rebuilding that corrupted database, using a process that I have to make up myself.  The day is mostly uneventful, but rereading my emails from Friday clearly shows that the stress from the past days and the anxiety about the DR test on Saturday have started to sharpen my language and harden my attitude in ways that are uncharacteristic of me.

I have another upgrade similar to Wednesday's, but this one goes very smoothly.  Circa thirty minutes.  Around 20 minutes before I leave for the day, our supervisor messages me that we may be postponing the DR test and watch for notification from him.  Hope mixed with aggravation.  I've been dreading this test because of the stress and hard work it requires and I want to just get it over with.  But I'd also love to be given my Saturday back, temporarily.  No word on this until 8:30pm, when he calls me to tell me things are still on.  He apparently has not called my manager but just me, asking me to notify them.  He does this a lot, treating me at times like a team lead, which is a nice stroke to my ego, but I still don't have any power, and realizing that second part is a hard crash.

The DR test turns out mostly OK, although it takes an hour longer than my estimate.  The things that broke were things we couldn't have anticipated without the test, and it feels good to have uncovered them so we can fix them.  I drop off books at the library, then attempt to run 12 miles, making it about 10 before my legs give out.  I guess a morning spent sitting in an office isn't the best warm-up.  Badly dehydrated, I spend a hot agonizing 3/4 hour at the Mid-Manhattan library, not finding what should be there, and arguing with the Express Checkout machine.  I have dinner alone and fight with my wife.

The week was bad, but like all bad things it's passed now.

In New York, Sometimes You Have To Earn A New Apartment

Partial List of Moments in Adulthood When I Felt Child-like Awe