Raid

1. Prepare new hard disk

Get a new hard disk preferably with the same LBA count.
Check the new hard disk for bad sectors and SMART errors.
Erase the partition table and data from the new hard disk.
Write down the serial number of the new hard disk.

2. Hardware RAID

2.1. Dell

2.1.1. Set up Hardware RAID 1

Note: This hardware raid set up method is require Legacy boot mode. After set up the hardware raid 1, you can change Legacy to UEFI boot mode.
Connect two harddisks to the computer with exactly the same size.
Boot up the computer.
Press Ctrl+I on Bios or RAID screen.
Select 1. Create RAID Volume and press Enter.
Select Volume1 and press Enter.
Select RAID1(Mirror) and press Enter.
Select 000.00 GB and press Enter.

Name:           Volume1
RAID Level:     RAID1(Mirror)
Disks:          Select Disks
Strip Size:     N/A
Capacity:       000.00 GB                       # Needed when disk size is different
Capacity:       223.6  GB
Sync:           N/A

Select Create Volume and press Enter.
Type y to confirm create this volume and press Enter.
Select Exit and press Enter.

2.1.2. Replace the faulty disk

Boot up the computer.
Press Ctrl+I on Bios or RAID screen.
Optional: Select Delete RAID Volume and press Enter.
Optional: Select Volume1:1.
Optional: Press the Delete key to delete the RAID volume.
Optional: Type y to confirm delete volume verification.
Optional: Press the Esc key.
Select New disk and press Enter.
Press Esc.
Type y to confirm the exit.
See intel-rapid-storage.adoc for software raid controller.

3. Software RAID

3.1. CentOS (Endian Firewall)

3.1.1. Information

Enter the following commands at a Command Line with root privileges.

cat /proc/mdstat
fdisk -l /dev/sda
fdisk -l /dev/sdb

hdparm -i /dev/sda
hdparm -i /dev/sdb

hdparm -I /dev/sda
hdparm -I /dev/sdb

/root/smartctl -a /dev/sda | grep -i 'reallocated_sector'
/root/smartctl -a /dev/sdb | grep -i 'reallocated_sector'

3.1.2. Replace faulty /dev/sda hard disk

Enter the following commands at a Command Line with root privileges.

hdparm -i /dev/sda      # Write down the serial number of the hard disk
hdparm -i /dev/sdb      # Write down the serial number of the hard disk
cat /proc/mdstat
mdadm --manage /dev/md1 --fail /dev/sda1
mdadm --manage /dev/md2 --fail /dev/sda2
mdadm --manage /dev/md3 --fail /dev/sda3
cat /proc/mdstat
shutdown -h now

Because the Endian Firewall is supposed to be up and running all the time it may boot as soon as the power is reconnected due to the BIOS setting Restore On AC Power Loss being set to On instead of Last State. Make sure to have the keyboard and monitor connected when reconnecting the power in order to be able to enter the boot menu and select the proper disk to boot from.

Clean the computer using the blower.
Replace the faulty hard disk (check the serial numbers).
Boot the computer from the 2nd hard disk using the boot menu.
Enter the following commands at a Command Line with root privileges.

hdparm -i /dev/sda      # Write down the serial number of the hard disk
hdparm -i /dev/sdb      # Write down the serial number of the hard disk
fdisk -l /dev/sda       # Make sure that /dev/sda is the new hard disk
fdisk -l /dev/sdb       # Make sure that /dev/sda is the new hard disk
sfdisk -d /dev/sdb | sfdisk /dev/sda    # Copy partition table from sdb to sda
shutdown -r now

Boot the computer from the 2nd hard disk using the boot menu.
Enter the following commands at a Command Line with root privileges.

fdisk -l /dev/sda
fdisk -l /dev/sdb
cat /proc/mdstat
mdadm --manage /dev/md1 --add /dev/sda1
mdadm --manage /dev/md2 --add /dev/sda2
mdadm --manage /dev/md3 --add /dev/sda3
cat /proc/mdstat

Wait for resync to finish.
Enter the following commands at a Command Line with root privileges.

cat /proc/mdstat
grub-install /dev/sda
grub-install /dev/sdb
shutdown -r now

3.1.3. Replace faulty /dev/sdb hard disk

Enter the following commands at a Command Line with root privileges.

hdparm -i /dev/sda      # Write down the serial number of the hard disk
hdparm -i /dev/sdb      # Write down the serial number of the hard disk
cat /proc/mdstat
mdadm --manage /dev/md1 --fail /dev/sdb1
mdadm --manage /dev/md2 --fail /dev/sdb2
mdadm --manage /dev/md3 --fail /dev/sdb3
cat /proc/mdstat
shutdown -h now

Clean the computer using the blower.
Write down the serial number of the new hard disk.
Replace the faulty hard disk (check the serial numbers).
Boot the computer from the 1st hard disk using the boot menu.
Enter the following commands at a Command Line with root privileges.

hdparm -i /dev/sda      # Write down the serial number of the hard disk
hdparm -i /dev/sdb      # Write down the serial number of the hard disk
fdisk -l /dev/sda       # Make sure that /dev/sdb is the new hard disk
fdisk -l /dev/sdb       # Make sure that /dev/sdb is the new hard disk
sfdisk -d /dev/sda | sfdisk /dev/sdb    # Copy partition table from sda to sdb
shutdown -r now

Boot the computer from the 1st hard disk using the boot menu.
Enter the following commands at a Command Line with root privileges.

fdisk -l /dev/sda
fdisk -l /dev/sdb
cat /proc/mdstat
mdadm --manage /dev/md1 --add /dev/sdb1
mdadm --manage /dev/md2 --add /dev/sdb2
mdadm --manage /dev/md3 --add /dev/sdb3
cat /proc/mdstat

Wait for resync to finish.
Enter the following commands at a Command Line with root privileges.

cat /proc/mdstat
grub-install /dev/sda
grub-install /dev/sdb
shutdown -r now

3.2. Debian (ANC Server)

3.2.1. Replace faulty /dev/sda hard disk

3.3. FreeBSD (pfSense Firewall)

3.3.1. Replace faulty ada0 hard disk

8               # Select "Shell"
bash
gmirror status

                    Name    Status Components
    mirror/pfSenseMirror  DEGRADED  ada1 (ACTIVE)

dmesg | grep ada

    ada0        Model: ST3160815AS      S/N: 6RX5NDSM   <-- BROKEN
    ada1        Model: ST3160815AS      S/N: 5RA7N6BC

exit
exit
6               # Select "Halt system"

Clean the computer using the blower.
Write down the serial number of the new hard disk.
Replace the faulty hard disk (check the serial number).
Boot the computer from the 2nd hard disk using the boot menu.
Enter the following commands at a Command Line with root privileges.
??? TODO ???

3.4. Windows

Software RAID 1 on Windows is possible but it is quite different from software RAID 1 on Linux and not intuitive at all. If you are not careful you may end up with a broken system and the only option left is a complete re-installation. Use BIOS or hardware RAID 1 instead. The Dell OptiPlex 9020, EX2 and EX3 computers allow RAID configuration in their BIOS.

Read the Configuring Disk Mirroring for Windows Server 2012.docx document.
See https://superuser.com/questions/948621/use-software-raid-1-drive-as-main-disk-and-boot-drive-in-windows-7
See https://superuser.com/questions/1005029/win-7-software-raid-1-primary-disk-failed-what-now

4. Rescue

4.1. Mount software raid partition

Enter the following commands at a Command Line with root privileges.

apt-get install mdadm
fdisk -l
cat /proc/mdstat
mdadm --assemble --scan
cat /proc/mdstat
mount /dev/md0 /mnt
vdir /mnt
umount /mnt