Yesterday evening I finished the big disk upgrade. As evidenced by my last post, I have been suffering through the death of some of IBMs old “Pixie Dust” disks. Both of them finally died and forced me into an earlier than planned disk upgrade. But this time I wanted to make sure that future upgrades would be easier. LVM and the Linux software raid tools have improved since my last big disk upgrade but were still horribly complicated. I dream of storage that provides raid 5 reliability, easy expandability (add a disk and the storage grows), and simple management. I guess I will have to wait for ZFS to make it’s way to Linux from Solaris. Apparently there are still some outstanding license restrictions that are holding this up and the port is being done through the FUSE layer.

Difficulties and rants aside, bunker is now up and running with 4 x 400GB disks sliced into 3 partitions (boot, main, testing). Boot is a simple partition at the start of each disk running in raid 1 (2 active, 2 spare), main is raid 5 with LVM on top and testing is a simple 5 GB at the end of each disk for, well, testing raid configurations like growing.

The LVM raid slice (1.1TB) is then setup with some operating partitions on top. The real difficulty came in setting up a initrd that would booth the LVM logical volume correctly. Part of this was that I was using an existing raid array and this confused the mkinirramfs program.

It’s done now and I can get on with my life knowing that I have reliable storage for mythtv recordings, mail and music.

I was awoken with a hangover and what sounded like an alarm in the background. Strange I don’t have one that sounds like that. I was hungover from Reitschule but not that hungover as to forget what my alarm sounds like. Next I thought it was my laptop beeping. No, that was next to me on the bed and in sleep mode. I got up, the head hurt even more and figured the building’s fire alarm must have gone off. The noise was coming from bunker, my main server. It was the sound of a disk drive continuously resetting:

recording of the disk dying in mp3 format

I shut of the power to the offending disk and it now rests in peace. 3 years of of a Cyrus mail spool have taken their toll.

Acutally I think it has been the heat of this summer that has been cruel to the disks. I thank the RAID gods every day for the way in which my disks have failed over so smoothly:

[simon@bunker[ttypts/4]~]$ sudo mdadm --misc --detail /dev/md2
/dev/md2:
Version : 00.90.03
Creation Time : Tue Aug  9 10:40:33 2005
Raid Level : raid1
Array Size : 1465024 (1430.93 MiB 1500.18 MB)
Device Size : 1465024 (1430.93 MiB 1500.18 MB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 2
Persistence : Superblock is persistent
Update Time : Fri Aug 11 06:39:36 2006
State : clean, degraded
Active Devices : 1
Working Devices : 1
Failed Devices : 1
Spare Devices : 0
UUID : b715e21a:0ac688c2:156d41bc:4263e71c
Events : 0.13856
Number   Major   Minor   RaidDevice State
0       0        0        0      removed
1      22        3        1      active sync   /dev/hdc3
2       3        3        -      faulty spare  /dev/hda3

I don’t like doing any big system changes before I go away and I am in Austria this weekend. I’ve been planning on doing the “big disk upgrade of 2006” afterwards, have the disks priced up and just need to pick them up from Bauers on Monday. The net result should be 1.2TB (4 x 400GB in RAID 5) of usable storage on the “/mnt/vault” array and 200 GB on “/” (3 x 200Gb in RAID 1 with hot standby). In the mean time I am going to hope that the single disk failed raid array holds up over the weekend.