A Tale of WWWoe
Some people have noticed that the Staggering Stories Forum has been rather, er, absent for the past few days. Why is it absent, I hear you ask. No, nothing I’ve done (for once). It is my long time host for the Staggering Stories site, PlusNet. They’ve accidentally deleted the forum. Again.
This will actually be the second time I’ve had to recreate the forum after they’ve wiped my data. Last year, on the 14th of April, they had some kind of disk failure and lost all their customers’ CGI sites. It was supposed to be routine maintenance. Before it was due to happen they stated they would back up all the data as a precautionary measure. Apparently it was during this backup that they suffered ‘multiple disk failures’. No, they didn’t have earlier backups to fall back on. It was all gone. They referred us all to their terms and conditions and washed their hands of it. Your data, you rebuild it. For a company that hosts other peoples data for them that was a pretty shoddy response. I didn’t have a backup of the forum, foolishly thinking I could trust them with my data, and spent a frantic weekend recreating it from scratch. The only mercy was that their MySQL database server, where all the forum posts were stored, wasn’t affected. I just had to recreate the program and theme…
I am probably missing an incident or two that didn’t affect me but the next I became aware of was in late November 2005. A script designed to check their CGI platform accidentally saw ‘a small number of sites removed in error’. Fortunately that time I wasn’t hit. Again they washed their hands of it and left the poor souls to rebuild again. They did, however, promise to address the issue to ensure this didn’t happen again.
Then rolls around the 9th of July 2006 and PlusNet famously loses over 700GB (yes Gigabytes!) of their customer’s emails. It transpires that one of their engineers accidentally reformatted a live disk array rather than the backup pack, ending up with two clean and shiny disk array packs! This cock-up was major headline news for technology news sites and it even made the BBC. Thanks to the publicity they at least tried to recover the data this time by sending the drives off to a professional data recovery firm but, alas, it was doomed.
After such a high profile loss of data you’d think they’d learn their lesson, wouldn’t you? Sadly not. On the 11th of August 2006, sometime between 07:54 and 12:20, they did it again. That pesky CGI maintenance script turned on PlusNet’s customers once more! It started deleting everyone’s CGI directories. Their engineers noticed pretty quickly this time and killed it but not before it had eaten its way through a good number of sites starting with A and B (mine being ajpurcell, sadly). So, there goes the Staggering Stories Forum, again. They claim that this time they did have backups. Unfortunately they had an error in that backup script too and some of those backups had failed, mine seemingly included. I say ‘seemingly’ because they haven’t actually told me yet. In fact my Customer Service Ticket has gone completely unanswered for four and a half days, not so much as a note saying ‘we’re investigating’. Nothing. They claim that only about 10 customer’s sites have failed to be restored. Not too many for them to inform us personally, is it?
I’m now in the process of moving both the Staggering Stories and my Adam Purcell domains to a new host – Bytemark. They offer more storage (4GB over 250MB!) and even a full Virtual Machine where I control everything. I install and configure the OS, the web server, the database, everything. They provide backup space on another machine but it is my responsibility to use it. They keep the machine physically up and running and with an Internet connection and leave me to manage the rest. No more relying on PlusNet clowns with their squirty flowers and hilarious scripts.
That being the case you may notice a bit of downtime with those domains as they are transferred across to Bytemark. There’s nothing much to be done about that. For now the restored Staggering Stories Forum is temporarily hosted at forum.freelancepeacekeepingagency.co.uk. It will shortly move to forum.staggeringstories.net but I’ll sing that from the rooftops when it happens!
On an unrelated and lighter note I just have to mention, with great pleasure, that Skepticality has returned! It’s a podcast and they describe themselves as ‘ Our podcast is here to bring you relevant, under reported current events, as well as in-depth discussions from a scientific, critical, skeptical, and humorous point of view. In our travels we will tackle the beasts of pseudoscience; the paranormal, supernatural, ufo / alien encounters, mis-understood history, astronomy, space, and overwrought legends – urban or otherwise. Welcome to Skepticality, truth in podcasting.’
Edit (17 Aug 2006): I see that The Register now has a news story on PlusNet’s latest problems, particularly the CGI deletions.
You’ve got to wonder how many times they will make the same mistake before they feel they’ve got it just right!
As always, buddy boy, the effort you put into the site is much appreciated.