FSCK Best Practices

From Thomas-Krenn-Wiki
Jump to navigation Jump to search

File system checks (FSCK) verify the consistency of file systems. Generally, a check of journaling file systems such as Ext3, Ext4, or XFS is performed quickly, since the metadata journal ensures consistency. Nevertheless, there may be situations in which a file system becomes corrupted and must be fixed. A regular FSCK is recommended on server systems to prevent file system problems.

Periodic check on Ubuntu

Ubuntu has deactivated the periodical file system check (FSCK) with the release of Ubuntu 12.04 Precise.[1]

  • Up through Ubuntu 12.04, FSCK was run at regular intervals during system boot. For this, max-mount-counts and interval-between-checks were set on certain values when generating the file system:
$ sudo tune2fs -l /dev/sda1 | egrep -i "mount count|Check interval|Last|Next"
Mount count:              2
Maximum mount count:      39
Last checked:             Mon Jun 29 20:23:44 2015
Check interval:           15552000 (6 months)
Next check after:         Sat Dec 26 19:23:44 2015
  • Starting on April 12, you will see the following output: a value of -1 for max-mount-counts and a value of 0 for interval-between-checks disable periodic FSCK:
$ sudo tune2fs -l /dev/sda1 | egrep -i "mount count|Check interval|Last|Next"
Last mounted on:          /
Mount count:              80
Maximum mount count:      -1
Last checked:             Fri Jul 18 21:27:32 2014
Check interval:           0 (<none>)

Hint: In server environments, it is generally recommended to verify file systems on a regular basis. Further information can be also found in:

Enable periodic FSCK

Attention: In some cases, it may be advisable to perform an FSCK manually rather than on a scheduled basis on server systems. For larger file systems, an FSCK can take quite a while. If a reboot causes the mount count to reach its limit or if you exceed the interval between checks, an FSCK will be performed during boot, during which the server will be unavailable! When performing the FSCK manually, you determine when it runs.

The use of LVM also allows to dismiss the FSCK on a snapshot. The downtime of the productive file system can be limited to a minimum with that. See also #Filesystem check with LVM Snapshot.

Requirement fstab

In fstab, the pass, or fs_passno, field must be set right for the periodic check.[2] The default value for the root filesystem is 1. It is best to set other filesystems to 2:

$ sudo vi /etc/fstab
# / was on /dev/sda1 during installation
UUID=410ffeb7-f200-4d44-9517-d1b0926bd574 /               ext4    errors=remount-ro 0       1
# /home was on /dev/sda2 during installation
UUID=30995b90-3348-4de3-905b-593f7e2de487 /home           ext4    defaults        0       2

Configure settings

To enable periodic FSCK at boot time, adjust the values max-mount-counts and interval-between-checks with tune2fs:

  1. max-mount-counts: Number of mount-processes on which the file system is verified automatically.
  2. interval-between-checks: Maximum time elapsed between two checks.
$ sudo tune2fs -c 60 /dev/sda1
tune2fs 1.42.9 (4-Feb-2014)
Setting maximal mount count to 60
$ sudo tune2fs -i 30d /dev/sda1
tune2fs 1.42.9 (4-Feb-2014)
Setting interval between checks to 2592000 seconds

Force FSCK when booting

Hint: The settings of the pass, or also fs_passno, field ( see #Requirement fstab) are ignored during a forced FSCK! All file systems listed in fstab are verified.

Up through Ubuntu version 15.10, it was possible to force an FSCK on the next reboot by creating the file /forcefsck. The next boot would then run an FSCK:

$ sudo touch /forcefsck

From Ubuntu 16.04, this is possible via the following kernel boot parameter (that can be set in the Grub configuration):[3]

fsck.mode=force

Auto repair

Attention: The following option triggers an auto-repair when errors occur. Automatic error correction is not always desirable or practical in every situation!

$ sudo vi /etc/default/rcS
[...]
# automatically repair filesystems with inconsistencies during boot
FSCKFIX=yes

Standard settings for new file systems

The standard settings for new created file systems can be made in the mke2fs.conf file. The periodical check of file systems is deactivated via default. To activate it, set enable_periodic_fsck to 1:

$ sudo vi /etc/mke2fs.conf
[defaults]
        base_features = sparse_super,filetype,resize_inode,dir_index,ext_attr
        default_mntopts = acl,user_xattr
        enable_periodic_fsck = 1
[...]

As a result, the FSCK is set up automatically for new file systems:

$ sudo mkfs.ext4 /dev/loop0
[...]
This filesystem will be automatically checked every 23 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

Manual file system check

A manual file system check for ext4 can be started with fsck.ext4 from the e2fsprogs package.[4] The name of the other tools, for example for XFS, can be found in the article Linux file systems.

Ideally, a FSCK is not only started on mounted file systems. The following is located in the manpage:[5]

Note that, in general, it is not safe to run e2fsck on mounted filesystems. The only exception is if the -n option is specified, and -c, -l, or -L options are not specified. However, even if it is safe to do so, the results printed by e2fsck are not valid if the filesystem is mounted.

If the file system is currently in a clean state, the FSCK must be started with -f:

$ sudo fsck.ext4  /dev/loop0
e2fsck 1.42.9 (4-Feb-2014)
/dev/loop0: clean, 11/7680 files, 2370/30720 blocks
$ sudo fsck.ext4 -f /dev/loop0
e2fsck 1.42.9 (4-Feb-2014)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/loop0: 11/7680 files (9.1% non-contiguous), 2370/30720 blocks

Not interactive filesystem check

The option -n performs a FSCK in read-only mode and assumes No as the answer to any question that may arise. The option is therefore suitable for not starting an FSCK interactively, for example regularly and automatized via cron job. Please note that the file system must not be mounted in this case.

Filesystem check with LVM snapshot

Using LVM allows you to perform file system checks using a snapshot. This means the device being checked does not need to be unmounted for the duration of the check. It is sufficient to unmount the file system being checked in order to create the snapshot. Theoretically, an LVM snapshot of a mounted file system would also be possible. However, the snapshot is only clean and truly consistent if the logical volume is not mounted.

This method for a FSCK using an LVM snapshot is also recommended by Theodore Ts'o in Mailing List Posts.[1][6]

Note: The lazy option can be helpful if users are currently working with the file system. With a lazy unmount, file system resources are released when they are no longer in use by users. You can also use lsof beforehand to check whether a file system is currently still in use. In the following example, unmounting is not possible because a user is still in the directory:

# lsof | grep -i mnt
bash       6963            root  cwd       DIR              252,0     1024          2 /mnt
lsof      14456            root  cwd       DIR              252,0     1024          2 /mnt
grep      14457            root  cwd       DIR              252,0     1024          2 /mnt
lsof      14458            root  cwd       DIR              252,0     1024          2 /mnt
# umount /mnt
umount: /mnt: device is busy.
        (In some cases useful info about processes that use
         the device is found by lsof(8) or fuser(1))

Attention: The snapshot must not fill up during the FSCK. Therefore, when creating the snapshot, make sure to choose an appropriate size! In this case, a snapshot size of 20 gigabytes was selected.

The following commands must be used for creating the snapshot and for starting the FSCK:

# umount -l /dev/vg00/data
# lvcreate -s -n data_snap -L 20G /dev/vg00/data
# mount /dev/vg00/data /mnt/
# fsck.ext4 -f /dev/vg00/data_snap
# lvremove -f /dev/vg00/data_snap

The advantage of LVM is that the file system can be remounted immediately after the snapshot is created.

References


Author: Christoph Mitasch

Christoph Mitasch works in the Web Operations & Knowledge Transfer team at Thomas-Krenn. He is responsible for the maintenance and further development of the webshop infrastructure. After an internship at IBM Linz, he finished his diploma studies "Computer- and Media-Security" at FH Hagenberg. He lives near Linz and beside working, he is an enthusiastic marathon runner and juggler, where he hold various world-records.



Author: Georg Schönberger

Translator: Alina Ranzinger

Alina has been working at Thomas-Krenn.AG since 2024. After her training as multilingual business assistant, she got her job as assistant of the Product Management and is responsible for the translation of texts and for the organisation of the department.


Related articles

Ext4 Filesystem
Linux file systems
Linux Software RAID Information