I’ve been having a terrible headache of crashing disk drives lately. For sure when this happens for twice in a row! Once at 2 march 2009, the second at 8 march 2009. This with the brand new HP DL 180 server hosted at Leaseweb.
Mental note: whatever you administrate:
KEEP BACKUPS READY! All time!
The first crash at 22:45 was swiftly taken care of, the disks were changed inside 24 hours with Western Digital, the dataloss was quite minimal. After working for 5 consecutive days all data was recovered, including my work for a second server which was ordered through Leaseweb. The restoration process was visible by miles, since the changes from ArtistBlog.ME/beta were implemented at once at the production side of ArtistBlog.ME.
The second crash happened just a day before writing this, this time at night. At 05:00 I’ve noticed the server was going very slow and I was able to send myself the latest snapshot of the backup which was ready to be transported. The server was brought back online at 18:00 thanks to Leaseweb support and after a bit of MySQL & permissions troubles the ArtistBlog.ME component was back on-line in no time later.
Leaseweb has responded me “the HP controller firmware seems to be buggy, which will be upgraded tomorrow morning.” Hopefully this firmware will fix the server with an automagical potion of fairy dust so it’ll be enchanted to the use of the ArtistPlug.ME network for many years. I’ve already demanded for a parallel solution if this server would corrupt it’s data one more time, to get a kosher prompt replacement of the server; because this could hurt customers with the second server i’ve been preparing at Leaseweb.
So, Leaseweb, let’s really hope so this HP DL 180 server will be healed up with the firmware for reliabilities sake…
update (19 june 2009): Until now the update has been working great! (holding wood). Leaseweb support has been swift and easy and the server has lived long and happy since that famous HP controller firmware update!
You’ll find a bit of reference material underhere!
