SATA hotswap drive in mdadm RAID array

I needed to replace a SATA drive in a mdadm RAID1 array and I figured I could try to do a hot swap. Before the step-by-step guide, this is how the system is set up for an orientation.

  • 2x1TB physical disks; /dev/sdb and /dev/sdc
  • Each drive contains one single partition; /dev/sdb1 and /dev/sdc1 respectively
  • /dev/sdb1 and /dev/sdc1 together make up the /dev/md0 RAID1 array

Here is what the array looks like:

$ cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid1 sdb1[0] sdc1[1]
976629568 blocks super 1.2 [2/2] [UU]

In the following steps we will remove one drive from the array, remove it physically, add the new physical drive and make mdadm rebuild the array.

Important note!
Since we’re removing one of the drives in a RAID1 set we do not have any redundancy anymore. If this is critical data on the array, this is the time to make a proper backup of it. The drive I’m removing is not faulty in any way and therefor I will not take any backups.

Step-by-step guide

  1. Mark the drive as failed
    $ sudo mdadm --manage /dev/md0 --fail /dev/sdb1
  2. Remove the drive from the array
    $ sudo mdadm --manage /dev/md0 --remove /dev/sdb1
  3. View the mdadm status
    $ cat /proc/mdstat
  4. If you prefer to shut down the system for a cold swap, do it now. Before the hot swap put the drive into standby with the following command
    $ sudo hdparm -Y /dev/sdb
    Make sure you know which drive you are going to remove before issuing this command. Operations to the disk will wake up the drive again.
  5. Remove the SATA signal cable first and then the SATA power cable.
  6. Mount the new drive and connect SATA power. I let the drive spin up for 5-10 seconds before connecting the SATA signal cable. If you did a cold swap, power on the system at this point.
  7. Identify the new drive and what device name it has. In my case, the new drive was conveniently named /dev/sdb, the same as the old one.
  8. Copy the partitioning setup from the other drive in the array to the new disk.
    Make sure the order is correct, otherwise we will erase the operational drive!
    $ sfdisk -d /dev/sdc | sfdisk /dev/sdb
  9. Add the new drive to the RAID array
    $ sudo mdadm --manage /dev/md0 --add /dev/sdb1
  10. The RAID array will now be rebuilt and the progress is indicated by the
    $ cat /proc/mdstat
    output. To have a more dynamic update of the progress use the following:
    $ watch cat /proc/mdstat

When the rebuild is done the status will look something like this again:

$ cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid1 sdb1[2] sdc1[1]
976629568 blocks super 1.2 [2/2] [UU]

Enjoy and be careful with your data!

8 thoughts on “SATA hotswap drive in mdadm RAID array

  1. Hi

    on step 2:
    2. Remove the drive from the array
    $ sudo mdadm –manage /dev/md0 –fail /dev/sdb1

    maybe:
    $ sudo mdadm –manage /dev/md0 –remove /dev/sdb1

    Am i right?

  2. Pingback: Replacing drives with mdadm | Craig Wright

  3. Thanks! That did the trick. I have never had to replace a raid 1 drive in years (more than 10 😉 of using them and was of course nervous about doing this.

  4. Ok, this is when you intervene before doning the hot swap. What if ie. someone just plugs the failed drive and then connects the new one while server is up and running? What would happen to array? What would be the correct procedure to add the new drive in array?

    • I just hot-swapped the new drive in and repeated steps 8-10 (copy partitions over to the new drive, then add back to the target array, then verify it’s in recovery).

      BTW, my compliments to the author of this article; helped get me finally squared away on my new system. 😉

Leave a Reply to John Cancel reply

Your email address will not be published. Required fields are marked *

 

This site uses Akismet to reduce spam. Learn how your comment data is processed.