This is starting to look bad for me… maybe Dragelet, Sivart & friends are correct about my technical skills
Basically, I managed to kill the LVM partition of my linux server. I noticed one of the TLR tarballs had a corrupt inode, so I attempted to do a FSCK of the file system.
Big mistake! The FSCK killed the ext3 journal, and blatted the root inode… everything now existed in the lost+found directory.
Not a problem I thought, I have a backup… of the database from 4 weeks ago… no file backup of the website though.
Time for a bit of lateral thinking… the data I need is still on the LVM, it’s just been orphaned… so all I need to do is copy the important file structures off onto a new system.
Bingo, rename the existing LVM partition (so I don’t accidently blat it!) and create a new XEN linux server. Mount the LVM partition within the XEN host, and selectively copy what I need over.
The tricky parts were:- 1) Remembering what packages I needed & 2) Importing the raw MySQL database files (otherwise 4 weeks of content was gone!)
Less than three hours later, this site is back up and running!
Tomorrow I’m gonna start looking into setting up a scheduled remote backup
Maybe I should also chase up another HDD and set-up a RAID1 mirror… so many possibilities!
P.S. The recent update to WordPress 2.1.1 appears to have eliminated all the Apache seg faults, another week of testing to confirm it’s gone, and I may be able to go back to my preferred theme!