mdadm: Modifying a RAID Array

Document Scope

Building on the guide "Creating a RAID Array", this guide will describe some different actions you can take to modify an existing RAID array built using mdadm. These will include stopping or starting the array, adding or removing disks, and (for some RAID levels) increasing the storage pool using a spare disk.

Prerequisites

To follow the steps in this guide, you will need:

• A non-root user with sudo privileges
• A Linux operating system (like Rocky Linux or Ubuntu) with the mdadm utility installed
• A basic understanding of RAID terminology and concepts. (To learn more about RAID and what RAID level is right for you, read our introduction to RAID article.)
• An existing RAID array
• An unused disk (if adding a spare or increasing your storage pool)
• For all commands naming a RAID array, we will be using /dev/md0 but your RAID may have a different number, so modify these commands as needed to match YOUR array

Stopping a running array

To stop an array, the first step is to unmount it. Step outside the mounted directory with the cd ~ command:

cd ~

Then unmount the device:

sudo umount /mnt/md0

Once unmounted, you should stop the array using the stop flag:

sudo mdadm --stop /dev/md0

Or you can stop all active arrays by running:

sudo mdadm --stop --scan

This will stop the array. You will have to reassemble the array to access it again.

Starting an array

To start a specific array, you can pass it in as an argument to mdadm --assemble:

sudo mdadm --assemble /dev/md0

To start all arrays defined in the configuration files or /proc/mdstat, run the following:

sudo mdadm --assemble --scan

If the correct definition for the array is missing from the configuration file, the array can still be started by passing in the component devices:

sudo mdadm --assemble /dev/md0 /dev/sda /dev/sdb /dev/sdc /dev/sdd

Once the array is assembled, it can be mounted as usual:

sudo mount /dev/md0 /mnt/md0

The array will now be accessible at the specified mount point.

Adding a Spare Device to an Array

Spare devices can be added to any arrays that offer redundancy, such as RAID 1, 5, 6, or 10. The spare will not be actively used by the array unless an active device fails. When this happens, the array will re-sync the data to the spare drive to repair the array to full health. Spares cannot be added to non-redundant arrays (RAID 0) because the array will not survive the failure of a drive.

To add a spare, pass in the array and the new device to the mdadm --add command:

sudo mdadm /dev/md0 --add /dev/sde

If the array is not in a degraded state, the new device will be added as a spare. If the device is currently degraded, the re-sync operation will immediately begin using the spare to replace the faulty drive.

Removing a Device from an Array

Removing a drive from a RAID array is sometimes necessary if there is a fault or if you need to switch out the disk. For a device to be removed, it must first be marked as “failed” within the array. You can check if there is a failed device by using mdadm --detail:

sudo mdadm --detail /dev/md0

Output for a three drive raid:

/dev/md0:
Version : 1.2
Creation Time : Wed Aug 10 21:42:12 2020
Raid Level : raid5
Array Size : 209584128 (199.88 GiB 214.61 GB)
Used Dev Size : 104792064 (99.94 GiB 107.31 GB)
Raid Devices : 3
Total Devices : 3
Persistence : Superblock is persistent

Update Time : Thu Aug 11 14:10:43 2020
State : clean, degraded 
Active Devices : 2
Working Devices : 2
Failed Devices : 1
Spare Devices : 0

Layout : left-symmetric
Chunk Size : 64K

Name : mdadmwrite:0 (local to host mdadmwrite)
UUID : bf7a711b:b3aa9440:40d2c12e:79824706
Events : 144

Number Major Minor RaidDevice State
0 0 0 0 removed
1 8 0 1 active sync /dev/sda
2 8 16 2 active sync /dev/sdb
0 8 32 - faulty /dev/sdc

Note the "State" reads degraded and the device /dev/sdc shows as faulty. This all indicates that a drive is no longer functioning.

If you need to remove a drive that does not have a problem, you can manually mark it as failed with the --fail option:

sudo mdadm /dev/md0 --fail /dev/sdc

Output

mdadm: set /dev/sdc faulty in /dev/md0

If you review the output of mdadm --detail, you will notice that the device is now marked faulty. Once the device is failed, you can remove it from the array with mdadm --remove:

sudo mdadm /dev/md0 --remove /dev/sdc

Output

mdadm: hot removed /dev/sdc from /dev/md0

You can then replace it with a new drive, using the same mdadm --add command that we demonstrated above. The array will begin to recover by copying data to the new drive.

Increasing the Number of Active Devices in an Array

It is possible to grow an array by increasing the number of active devices within the assembly. The exact procedure depends slightly on the RAID level you are using.

With RAID 1 or 10

Begin by adding the new device as a spare, as it was demonstrated above:

sudo mdadm /dev/md0 --add /dev/sdc

Find out the current number of RAID devices in the array:

sudo mdadm --detail /dev/md0

Output

/dev/md0:
Version : 1.2
Creation Time : Wed Aug 10 15:29:26 2016
Raid Level : raid1
Array Size : 104792064 (99.94 GiB 107.31 GB)
Used Dev Size : 104792064 (99.94 GiB 107.31 GB)
Raid Devices : 2
Total Devices : 3
Persistence : Superblock is persistent

In this example, the array is configured to actively use two devices. It does reveal, however, that the total number of devices available to the array is three because of the spare.

Now, reconfigure the array to have an additional active device. The spare will be used to satisfy the extra drive requirement. Remember to replace your target number of raid devices in this command. Here we are increasing a raid 1 with 2 devices to 3. If you’re in raid 10 with 4 devices, and have the additional drive, increase it to 5:

sudo mdadm --grow --raid-devices=3 /dev/md0

The array will begin to reconfigure with an additional active disk. To view the progress of syncing the data, run the following:

cat /proc/mdstat

You can continue to use the device as the process completes.

With RAID 5 or 6

Begin by adding the new device as a spare as demonstrated above:

sudo mdadm /dev/md0 --add /dev/sdd

Find out the current number of RAID devices in the array:

sudo mdadm --detail /dev/md0

Output

/dev/md0:
Version : 1.2
Creation Time : Wed Oct 5 18:38:51 2022
Raid Level : raid5
Array Size : 209584128 (199.88 GiB 214.61 GB)
Used Dev Size : 104792064 (99.94 GiB 107.31 GB)
Raid Devices : 3
Total Devices : 4
Persistence : Superblock is persistent

In this example, the array is configured to actively use three devices, and that the total number of devices available to the array is four because of the added spare device.

Now, reconfigure the array to have an additional active device. The spare will be used to satisfy the extra drive requirement. When growing a RAID 5 or RAID 6 array, it is important to include an additional option called --backup-file. This will point to a location off the array where a backup file containing critical information will be stored:

Note: The backup file is only used for a very short but critical time during this process, after which it will be deleted automatically. Because the time when this is needed is brief, you will likely never see the file on disk, but in the event that something goes wrong, it can be used to rebuild the array. This post has some additional information if you would like to know more.

sudo mdadm --grow --raid-devices=4 --backup-file=/root/md0_grow.bak /dev/md0

The following output indicates that the critical section will be backed up:

mdadm: Need to backup 3072K of critical section..

The array will begin to reconfigure with an additional active disk. To view the progress of syncing the data, run:

cat /proc/mdstat

You can continue to use the device as this process completes.

After the reshape is complete, you will need to expand the filesystem on the array to utilize the additional space:

sudo resize2fs /dev/md0

Your array will now have a filesystem that matches its capacity.

With RAID 0

RAID 0 arrays cannot have spare drives because there is no chance for a spare to rebuild a damaged RAID 0 array. You must add the new device at the same time that you grow the array.

First, find out the current number of RAID devices in the array:

sudo mdadm --detail /dev/md0
Output
/dev/md0:
Version : 1.2
Creation Time : Wed Aug 10 19:17:14 2020
Raid Level : raid0
Array Size : 209584128 (199.88 GiB 214.61 GB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent

You can now increment the number of RAID devices in the same operation as the new drive addition:

sudo mdadm --grow /dev/md0 --raid-devices=3 --add /dev/sdc

You will receive output indicating that the array has been changed to RAID 4.

mdadm: level of /dev/md0 changed to raid4
mdadm: added /dev/sdc

This is normal and expected. The array will transition back into RAID 0 when the data has been redistributed to all existing disks. You can check the progress of the action by running:

cat /proc/mdstat

Once the sync is complete, resize the filesystem to use the additional space:

sudo resize2fs /dev/md0

Your array will now have a filesystem that matches its capacity.