Sunday, February 21, 2010
Have a contingency plan and know when to activate it.
I worked in an Information Technology department for many years, and this was one of our mantras. So it's been interesting to deal with contingency planning myself over the past two days.
Here's what those past two days have looked like for me, with approximate times:
Saturday, 6 p.m.
I came home from working with a client, went to my computer, and checked e-mail - and when no messages came in, I knew something was wrong. I went to my web site - and there was no web site! Horrors! But I could browse the web just fine.
So I called the company that hosted my web site, and was told there was a significant data center problem, but everything would be fixed by midnight. I wasn't happy - that would mean almost 11 hours of downtime, since the problem had started a bit after 1 p.m. - but I went to bed confident that all would be fine in the morning.
Sunday morning, 8 a.m.
Still no e-mail, no web site. Now I was told things should be fixed by 7 p.m. The phone support person did all he could do - he acknowledged the problems this extended downtime caused for me, and gave me the best information he had - but that didn't make the downtime situation any better.
Sunday morning, 8:30-noon:
No longer trusting the estimates, and unable to accept this extended downtime, I went searching for a new hosting company that met my requirements, which include around-the-clock phone support in the U.S. Fortunately, I already had some leads that I'd noted from prior reading.
I placed my order with my new hosting company.
Sunday, early afternoon:
I did all the technical work required to get my site moved and e-mail running again. Fortunately, I understood what needed to happen, and that went pretty smoothly. There was one glitch, but having a friend test my new site uncovered that one.
Sunday, 3 p.m.
My site came back up, at least for some users. E-mail started flowing again.
Sunday, 5:00 p.m.
My new hosting company sent me a welcome message on Twitter.
Sunday, 5:15 p.m.
My original hosting company provided updated information; service was now expected to be restored by 7 a.m. on Monday.
So what did I learn from my web site hassles? Being organized for such an eventuality really helps! Here are the basics of being prepared:
1. Have a plan.
I could have saved about four hours if I knew exactly what hosting company I would move to if mine ever had a catastrophic failure. But having some leads was much better than starting from scratch.
2. Review and modify the plan.
While I'll pick a "next in line" hosting company now, I know I need to review that decision periodically; companies change, for better and worse.
3. Know when to act on the plan.
While I hadn't put my decision criteria into words, I think I made some good choices here. I made the leap when:
- I no longer trusted the information coming from my old company.
- The probable downtime from staying put was long enough that the probable downtime associated with moving seemed like the lesser of the evils.
And these same lessons apply to other areas of our lives - which can be much more important then a web site problem. For example, I've had similar issues when one of my cats got sick on a holiday weekend. Where do I go when my vet is closed? When do I decide it's time to go, rather than wait for my normal vet to be open?
We know we need plans to deal with major issues - a fire, a serious earthquake or hurricane, a medical emergency. (We may not have those plans, but we're aware of the need!) But it's easy to overlook the more mundane blips in our lives, and make sure we're reasonably ready for them, too.