ext4 options for a media drive

2016 update: I’ve published a new article about setting up a new drive on my NAS. If you are doing this now I’d highly recommend you use a filesystem that has built in bit-rot protection such as BTRFS or ZFS. Take a look at that article for instructions of how to do that!

At the moment I am reinstalling my desktop PC, and setting it up to be a media server / HTPC. One of the things that has baffled me is the best options to use for ext4 on the media partition. A quick search revealed nothing definite, hence this post. It is mainly as a reminder for when I reinstall (although I’ll probably use brtfs when it is more stable), however hopefully someone else will find it useful!

Intro

I have a 1TB drive which I use as a media drive. I currently have it partitioned as follows:

# fdisk /dev/sdb -l

Disk /dev/sdb: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xffffffff

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1         608     4883728+  83  Linux
/dev/sdb2             609      121601   971876272+  8e  Linux LVM

Yeah nothing fancy, a small 5GB partition at the start and the rest setup in LVM. The small partition is to serve as a simple recovery partition if I want to access the contents of the drive on another OS; I can do so just by booting that up in a virtual machine. 5GB is probably a bit big, however it isn’t as if space is scarce.

In terms of LVM I just have a volume taking up the full size of the disk:

# lvdisplay
  --- Logical volume ---
  LV Name                /dev/vg/media
  VG Name                vg
  LV UUID                2OJIN6-sAx8-KRIz-SNvl-RU0I-2Xgl-nzUH4V
  LV Write Access        read/write
  LV Status              available
  # open                 0
  LV Size                926.85 GiB
  Current LE             237274
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           254:0

I have previously setup different partitions for certain things, however in the end that just got a pain when one partition got too small. This setup kind of defeats the point of LVM, however if I want another partition (e.g. to dual boot) or to add another disk I can easily do so.

ext4

Using the default options, on the face of it, ext4 doesn’t provide any significant advantages over ext3, or even ext2 when used for a media drive. However there are some nice features behind the scenes:

• Larger maximum filesystem size (ext3 is limited to 16TB)
• Extents (a set of contiguous blocks; improves performance and leads to decreased fragmentation)
• Multiblock allocation (as opposed to single block allocation; improves write performance for large files)
• Delayed allocation (doesn’t write the file to disk immediately, waiting until more of the file is in the cache, and then allocating a larger number of blocks; leads to decreased fragmentation)
• Faster fsck (Try running fsck on a 1TB ext2 volume ^_^)
• Persistant preallocation (Reserves a large number of blocks, which aren’t going to be used yet but will be at some point; on par with what modern P2P applications do by filling a file with zeros, however implemented into the filesystem so much more efficient)

A good introduction to ext4 (which these were taken from) is the ext4 article on Kernel Newbies.

Creating the filesystem

The options for mkfs.ext4 are basically unchanged from ext3, in order to enable the high performance options most of the work will be done in tune2fs and mount options.

# mkfs.ext4 -b 4096 -M 0 -L Media /dev/vg/media

The b option sets the block size (this will most likely default to 4096 bytes), the M options set the percentage of space reserved for the super user, and the L option sets the filesystem label. This command shouldn’t take more than a couple of minutes to run.

Optimising

Next up is to pass some options to tune2fs to enable the advanced features. These options could have been passed to the previous command, however to easily explain they have been listed separated. You can also run this command to upgrade an existing ext3 file system. A lot of these are enabled by default, however they are listed here for completeness sake.

• has_journal – Enables the ext4 journal
• extent – Enables extents (see above)
• huge_file – Enables files over 2GB in size
• flex_bg – Allows inode metadata to be placed anywhere on the partition (as opposed to traditionally at the start)
• uninit_bg – Enables new blocks to not be initialised and enables block checksumming. Significantly speeds up creation and fsck
• dir_nlink – Enables unlimited sub directories
• dir_index – Enables a hashed B-tree for linking subdirectories

# tune2fs -O has_journal,extent,huge_file,flex_bg,uninit_bg,dir_nlink,dir_index /dev/vg/media

As this just enables various options on the file system it will execute rather quickly, especially if the file system is empty. Note that if you are migrating an existing partition, most of these options will only apply to newly created files.

Mount options

The real killer performance can be obtained by setting certain mount options. Note that these seriously increase the likelihood of corruption if a system crash occurs, however in the case of a media drive, where there are a limited number of writes, I don’t feel that this is an issue.

• barrier=0 – Disables a protection option that ensures everything is written to the journal before being committed, however decreases performance by ~30%; this is the default on the majority of systems
• commit=60 – Only commits the journal to disk every 60 seconds
• noatime – Disables the access time attribute which causes the metadata to be updated every time a file is accessed
• data=writeback – Enables journaling of metadata only; can cause files to become corrupt if a system crash occurs
• journal_async_commit – Write the journal to disk asynchronously

These can be set in your /etc/fstab, e.g.:

/dev/vg/media  /media  ext4    defaults,user,barrier=0,commit=60,noatime,data=writeback,journal_async_commit   0   0

So thats it! If you find any other options that you deem useful, feel free to post them in the comments!