We had a website disaster this past weekend.
We have several sites: Divorce Discourse, of course, plus a site for our family law practice. We also have Stay Happily Married and a few others. They’re all hosted by a company that specializes in providing website services for businesses.
On Friday afternoon, we got an e-mail from our hosting provider explaining that it planned to “upgrade” us to a more powerful server. The e-mail indicated that it would notify us in 48 to 72 hours as to when the work would proceed and what steps we would have to take.
The Surprise “Upgrade”
Sunday, while I was window-shopping in the Polanco neighborhood in Mexico City, my phone went crazy with e-mails notifying me that our sites were down. Our host hadn’t yet sent any notification that a change was underway. Mild panic ensued.
I rushed back to the apartment where I’m staying and met up with our operations guy, who communicated with our web host and determined that it was, in fact, doing the upgrade even though it hadn’t warned us. The sites were down, and the hours started ticking away.
After eight hours, our sites were still down. We communicated with the host, which told us that things were normal and proceeding. At 2 AM, the company finally acknowledged that things were not normal, so we decided to take action. We didn’t want our usual Monday morning crowd of several thousand site visitors to get a “Site Not Available” message. We proceeded to restore our sites on another host and pointed our traffic to the new location. Our operations guy had us up and running a couple of hours later.
Learning From Our Mistake
Because our sites were down, we couldn’t simply migrate the old data to the new host. We had to restore our files from a backup copy. That’s when we discovered that we didn’t have a backup of everything. We had much of what we needed, but we were missing several key pieces of data.
We realized that we might never recover the missing data if the “upgrade” didn’t go as planned. That’s when we shifted gears and got into “NEVER AGAIN” mode. We focused on fixing our backup problem while the issue was fresh for us. We spent much of Monday morning working out a more comprehensive backup solution.
Then we got some good news. Our old host finally worked its way through the upgrade and overcame the issues it encountered. The host was able to provide us with the data that we had failed to back up.
This time, the disaster was averted.
Failing to Plan Is Planning to Fail
I’m sure you’re taking care of backing up the data on your personal computers and phones. You’re probably also doing a good job of backing up your servers (if you still have any).
Be sure, however, that you’re getting it all. Figure out whether you’re really backing up everything. Are you getting every last bit and byte?
Go further. Now’s a good time to be sure it’s all working. This is the perfect time to check and see whether everything is included. This is a perfect time to be sure you’re prepared in case one of your cloud-based providers drops the ball.
Take a few hours and pretend that your data is gone. Think about what that might mean for your business.
- What will you do?
- Will you solve the problem in a few hours, or will it take a few weeks?
- Does someone know what to do and how to do it?
- Do you know who to call and how to proceed?
Don’t get caught without a proper backup like we did last weekend. Be sure you’re ready if disaster strikes.