Setting up a new drive on your Linux NAS

I received some new drives recently to expand my NAS. Here I document the process of setting up a new drive, mainly for my future reference, to get everything in one place so I don’t need to look at a load of different articles to figure out what to do.

Write down the details

When I first receive a drive the first thing I do is update a spreadsheet with the details of it. This consists of:

• Manufacturer
• Model
• Capacity
• Serial number (double check this is correct!)
• Warranty expiry date
• A unique identifier in the format , e.g. 001wd3
• The machine and bay it is mounted in

Once you’ve inserted it into your machine you need to figure out the device identifier, which is something like sdX – where X is a letter (it may also be a different format, depending on the drive interface). This probably won’t map to physical bank or slot, so you need to figure out which drive is which. In my case I’m adding two new drives which are the same size and model. The lsblk command is the first step to figuring it out:

$ lsblk
NAME              MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda                 8:0    1  2.7T  0 disk
└─vg001wd3-pool0  254:0    0  1.5T  0 lvm  /pool
sdb                 8:16   1  2.7T  0 disk
sdc                 8:32   1  2.7T  0 disk
sdd                 8:48   1 58.7G  0 disk
└─sdd1              8:49   1    1M  0 part

You need to look for the drives which don’t have any partitions associated (assuming you drive is brand new), so in this case sdb and sdc. You can then use smartctl to check the serial number:

$ smartctl -i /dev/sdb
smartctl 6.4 2014-10-07 r4002 [x86_64-linux-4.6.0-0.bpo.1-amd64] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Seagate NAS HDD
Device Model:     ST3000VN000-1HJ166
Serial Number:    W6A1JKW8
LU WWN Device Id: 5 000c50 09bbbe2bf
Firmware Version: SC60
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5900 rpm
Form Factor:      3.5 inches
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2, ACS-3 T13/2161-D revision 3b
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:    Sat Jul  9 07:54:51 2016 EEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

Drive Burn In

Before putting the drive in production you should perform a series of tests to ensure that it is working. This consists of running the passive SMART tests, and active badblocks test. If the drive fails this ask for a replacement from your retailer (don’t RMA it, as you’ll get a refurbished drive).

First run a couple of quick SMART tests, you’ll need to wait for the test to finish on a drive before running the next test, but you can test multiple drives at once. After running each command it’ll tell you how long the test will take (only a few minutes), and if you try to run multiple tests at once it’ll tell you that one is in progress.

$ smartctl -t short /dev/sdb
$ smartctl -t conveyance /dev/sdb

Next up is the badblocks test. The SMART tests are passive, so they don’t usually report anything until the drive has attempted to be used. The badblocks test will write a set pattern to the whole drive, then read it back and check it is correct. This will wipe the drive, so don’t do this if you want to keep the data on it!

$ badblocks -b 4096 -c 98304 -ws /dev/sdb

The -b 4096 parameter sets the block size of the drive (which is 4096 for modern hard disks), and the -c 98304 parameter says to read 98304 blocks at once which is faster – this doesn’t affect the integrity of the test but will use more memory. In my case each instance of badblocks (I was setting up two new drives at once) used around 2GB of RAM. Note that this test will take a while, so run it in screen or tmux. In my case it took 48 hours on a 3TB drive.

Once that is complete, run a long SMART test which will take another few hours:

$ smartctl -t long /dev/sdb

Once complete, you can then check the SMART parameter of the drive.

$ smartctl -a /dev/sdb

The important bits to check are:

• The test log should show no errors (“Completed without error”)
• The “Current_Pending_Sector” and “Offline_Uncorrectable” parameters should be zero

Depending on who you ask the SMART parameters can mean a million things (most not important), so I’ll leave this as an exercise for the reader to figure out.

Full Disk Encryption

If you are going to setup full disk encryption now is the time. In my case I won’t be storing anything I need to keep secure (e.g. business documents) on this, but I’d like to use it as if I ever need to sell or RMA a drive I won’t have to worry about the data being read (I don’t really want some random person getting hold of my family photos). Unfortunately the CPU on my NAS is too slow, so I am not using it.

You can run the following command to test the performance of decryption operations on your CPU:

$ cryptsetup benchmark
# Tests are approximate using memory only (no storage IO).
PBKDF2-sha1       346751 iterations per second
PBKDF2-sha256     193893 iterations per second
PBKDF2-sha512     153121 iterations per second
PBKDF2-ripemd160  230760 iterations per second
PBKDF2-whirlpool   73801 iterations per second
#  Algorithm | Key |  Encryption |  Decryption
     aes-cbc   128b    74.7 MiB/s    87.2 MiB/s
 serpent-cbc   128b    35.8 MiB/s    97.8 MiB/s
 twofish-cbc   128b    81.5 MiB/s   107.3 MiB/s
     aes-cbc   256b    58.5 MiB/s    65.6 MiB/s
 serpent-cbc   256b    39.7 MiB/s    97.5 MiB/s
 twofish-cbc   256b    86.1 MiB/s   106.5 MiB/s
     aes-xts   256b    81.2 MiB/s    83.5 MiB/s
 serpent-xts   256b    90.4 MiB/s    92.4 MiB/s
 twofish-xts   256b    99.2 MiB/s    98.2 MiB/s
     aes-xts   512b    62.1 MiB/s    64.1 MiB/s
 serpent-xts   512b    91.0 MiB/s    92.5 MiB/s
 twofish-xts   512b    99.3 MiB/s    98.3 MiB/s

The drives I have can each read at 150MB/s – 200MB/s, so with four in the NAS, it can potentially read at 800MB/s. It only has gigabit ethernet, so in reality I’ll only be doing reads at around 125MB/s. However, housekeeping operations like scrubs will be faster with higher read speeds.

In reality I’d like to get at least 200MB/s decryption speeds for full disk encryption to be worth it for me, but as you can see the CPU can barely achieve half that (while idle). At this speed it’ll just be too slow, as the CPU is old (6 years) and doesn’t have native encryption instructions. For comparison my laptop, which has a modern Intel Core i5 processor, achieves 350MiB/s encryption and 1900MiB/s decryption.

As I’m not using it, I won’t explain how to setup full disk encryption, but the Arch wiki has good instructions (even if you are using another Linux distribution).

LVM

On top of the physical device, or encrypted device, you should create a LVM volume:

$ pvcreate /dev/sdb
$ vgcreate 002sg3 /dev/sdb
$ lvcreate -l 100%FREE 002sg3 -n pool0

In this case 002sg3 is the identifier I am using for this drive, and pool0 is the identifier I’m using for the device pool. By using LVM it means if there is ever an issue with a drive, the filesystem tools will show your identifier instead of the system’s identifier – so you can easily figure out what drive is broken.

In this case we aren’t really using any features of LVM, but there is basically no performance impact, and it’s worth it for future expandability (and being able to easily identify a drive when it fails!). You could use it to, for example, split a drive amongst two pools, one in RAID1 and another in RAID6.

Filesystem

Next up create or add the drive to your filesystem pool. I’m using BTRFS, so to create a new pool it’s this:

$ mkfs.btrfs /dev/002sg3/pool0

And to add it to the existing pool (there is currently a single drive, I want to turn it into RAID1), it’s this:

$ btrfs device add /dev/002sg3/pool0 /pool
$ btrfs balance start -dconvert=raid1 -mconvert=raid1 /pool

(/pool in this case is the mount point of the filesystem – BTRFS can perform these operations while the filesystem is live!)