Slint with BTRFS

In this article we outline the unique characteristics of the BTRFS file system and how a Slint system is configured at installation to take advantage of them.

In order not to overload this document with definitions, we refer the reader to the glossary included in the Slint Manual and more specifically for btrfs "jargon" to the glossary included in its documentation.

A BTRFS file system consists of a logical volume that can span multiple block devices (partitions or disks). Initially Slint is installed on a single partition.

A subvolume is a subtree of files inside the volume, the root of which can be mounted as if it were an independent filesystem. However, the space allocated to a volume is also allocated to each of its sub-volumes: it is therefore not necessary to distribute it between the sub-volumes, as if they were in distinct partitions.

Subvolumes can be created at the same time as the volume (by the mkfs.btrfs command), but also added or deleted later using the btrfs tools.

As an example the commands below create the Slint "system" volume and its subvolumes, in case BTRFS is used. In the following $ROOTNAME is the name of the partition in which Slint will be installed and $SLINT the mount point of the system volume during installation.

mkdir /SLINT
SLINT="/SLINT"
mkfs.btrfs -L root $ROOTNAME
mount -o compress=zstd:3,noatime $ROOTNAME $SLINT
btrfs subvolume create $SLINT/@
btrfs subvolume create $SLINT/@home
btrfs subvolume create $SLINT/@swap
umount $SLINT
mount -o subvol=/@,compress=zstd:3,noatime $ROOTNAME $SLINT
mkdir $SLINT/home
mkdir $SLINT/swap

The $ROOTNAME partition (denoted by its UUID which we will call <uuid>) will then be mounted three times (once per subvolume) each time Slint is started, as indicated in the /etc/fstab file:

UUID=<uuid> / btrfs subvol=/@,compress=zstd:3,noatime 0 0
UUID=<uuid> /home btrfs subvol=/@home,compress=zstd:3,noatime 0 0
UUID=<uuid> /swap btrfs subvol=/@swap,compress=zstd:3,noatime 0 0

Which gives for example (taken from the output of lsblk /dev/sda5):

Filesystem      Size  Used Avail Use% Mounted on
/dev/sda5        50G   22G   28G  44% /
/dev/sda5        50G   22G   28G  44% /home
/dev/sda5        50G   22G   28G  44% /swap

We see that the space available for the volume (28G) is also available for each of the sub-volumes. On the other hand, the "compress_zstd:3" mount option means that all the files that will be stored will be compressed by the zstd utility with compression level 3. Thus the size of the system is approximately half that with the ext4 file system.

If we mount the device at the root of the volume (without mention of a subvolume), like this:

mount /dev/sda5 /mnt

we see the subvolumes appear as subdirectories of /mnt:

ls /mnt
@ @home @swap
ls /mnt/@home/didier
Desktop Documents GNUstep Images Templates Music Audience Downloads Videos

Another remarkable feature of BTRFS is "copy-on-write". When editing a file, modified parts are written to another location, the metadata (which record the location of all parts of the file) being updated. As long as the previous data are still referenced, they will stay untouched.

This makes it very easy to create snapshots of BTRFS subvolume: taking a new snapshot is "free", as it uses very few disk space: only the metadata of the snapshot are written, which stores the physical location of all parts of files in the subvolume, and this is done almost instantly. The space used by the snapshot will only grow when the original subvolume and the snapshot differ, as then the snapshot will retrieves data removed from the original. On the other hand data added to the original will not be included in he snapshot: in other words a modification of the original subvolume does not modify the snapshot.

Slint includes the utility absm, which takes snapshots of the subvolume @, allowing to return to a previous state of the system if an update goes wrong, by booting the system on this snapshot, selected in the GRUB boot menu. To know its usage, just type absm as root or using sudo.

BTRFS also verifies the integrity of each file when it is opened using a checksum, making it unnecessary to check the file system at system startup. In addition, the "btrfs scrub" command verifies the integrity of all files, including those that are rarely read. By default, the utility btrfsmaintenance included in Slint runs "btrtfs scrub" once a week.

btrfsmaintenance also triggers "btrfs balance" once a week by default. This command allows you to distribute the data between the devices if the file system extends over several, but also to reorganize the space used, in particular to free the unallocated space of the file system, which improves in particular the performance of BTRFS in the case of a hard disk. To know more about btrfsmaintenance read /usr/doc/btrfsmaintenance*/README.html

Other useful and BTRFS-compatible tools are included in Slint, all have a --help option and a man page:

  • jdupes detects duplicate files and allows you to take action accordingly, regardless of the file system used

  • btdu allows to know with precision the place occupied by the BTRFS sub-volumes and directories

  • restic is a very versatile backup software that can be used for any file system.

Recommendations.

  • BTRFS needs room to breathe and reorganize. Always keep 10-20% free space.

  • Some tool options included in the btrfs-tools package can be dangerous or counterproductive. Use only those whose effect you know perfectly well and if in doubt seek advice first, on the Slint mailing list or by IRC on irc.libera.chat, channel #btrfs.

  • In particular, avoid using "btrfs filesystem defrag" and especially not "btrfs check --repair".

  • Before using a command, carefully read the corresponding man page (starting with "man btrtfs" which lists the others).

  • As with any file system: make regular backups! This is usually the only way to recover your data in the event of hardware failure.

For further:

Customize a Slint system layout

Introduction

In this article, we review how the Slint installer organizes the system’s basic directories (their layout), and show how to customize this layout after installation. The installation process is detailed in Chapter 2: Installation of the HandBook.

The installer places all the directories listed below, which constitute the core of the system, in the same partition:

bin boot dev etc home lib lib64 media mnt proc root run sbin srv sys usr

If other partitions already formatted are accessible during the installation, the installer proposes that they be mounted at system startup, with a directory name chosen by the user as mount point, except those listed above.

Furthermore, if automatic partitioning has been chosen, the installer proposes to create an additional partition in the remaining space available on the device, with a mount point chosen by the user, also outside those listed above.

In sum, the installer:

  • places all directories constituting the core of the system in the same partition,

  • allows to set up other directories in other partitions, previously formatted.

However, you may want to install files or directories that are part of the core of the system such as /home in another partition, especially if space is limited on the device. For example, if the computer is equipped with a fast device (SSD or NVMe) and also with a slower but larger capacity hard disk, you might want to install the system on the SSD or NVMe and the larger files on the hard disk.

You may also want to share large files usually stored in /home with another system, such as images, various documents, audio or video files.

Practical guide

As an example we show how to move /home to another device, with some variations.

Move /home entirely to another device

We assume that you want to move the /home directory, initially installed on an SSD or NVMe, to a hard disk. This can be done right after installation or later. All the following is to be done as root.

First, you need a formatted partition on the hard disk, named /dev/sdb1 below, to host /home:

  • Create if not already done on the hard disk the partition table (GPT type) and the partition using one of theses tools: parted, gdisk, cgdisk or gparted, all included in Slint, big enough for its intended use.

  • Format this partition using one of the btrfs, ext4, or xfs file types (commands mkfs.btrfs, mkfs.ext4 or mkfs.xfs)

To be able to copy the /home directory to this partition you need to mount it, for example on /mnt, with one of the following commands:

mount /dev/sdb1 /mnt -o compress=zstd:3 # if /dev/sdb1 has been formatted with btrfs
mount /dev/sdb1 /mnt # if /dev/sdb1 was formatted with xfs or ext4

Then copy the files, for example like this:

cp -a /home/* /mnt

or like this:

rsync -aAXv /home/* /mnt

Then, check that the copy has been done correctly:

diff -r /home /mnt

Then modify the /etc/fstab file to mount /home on the new partition at the next boot.

If the root file system is btrfs, /home is mounted as subvolume. In this case comment out the line in the file used to mount /home by inserting a # character at its beginning. For example edit this line (replace <uuid1> with the UUID value specified in /etc/fstab):

UUID=<uuid1> /home btrfs subvol=/@home,discard=async,compress=zstd:3,noatime 0 0

which becomes:

#UUID=<uuid1> /home btrfs subvol=/@home,discard=async,compress=zstd:3,noatime 0 0

Then insert in the file a line to mount at boot time /home on the new partition. We will designate <uuid2> the value of its UUID, displayed by this command:

lsblk -lno uuid /dev/sdb1

The line to insert in the /etc/fstab file depends on the chosen file system of /dev/sdb1.

For btrfs:

UUID=<uuid2> /home btrfs compress=zstd:3,discard=async,noatime 0

For ext4 or xfs:

UUID=<uuid2> /home ext4 noatime 1 2

Then, reboot the machine. In case of problem, to go back:

  • Case occurring uncomment the commented line in /etc/fstab (remove the # character at the beginning of the line)

  • Comment the new one (insert a # character at the beginning of the line)

  • Then reboot.

Once the new partition has been checked for correct operation, the old one can be deleted. The way to do this depends on the file system configured by the installer. Once the machine is rebooted, make sure that /home is mounted on the new partition, for example with the command

findmnt -o /source

Then delete the old /home:

  • In case of btrfs type these commands (we assume that the system was installed in /dev/sda3):

    umount /mnt
mount /dev/sda3 /mnt -o subvolid=0
btrfs subvolume delete -c /mnt/@home
btrfs subvolume sync /mnt # this command initiates the recovery of the space occupied by the old /home
btrfs filesystem sync # this command ends the recovery of the space occupied by the old /home
umount /mnt

  • In case of ext4 or xfs, we need that the new /home be not in use to remove the old one. So:

    First, reboot in text mode.
Login as root # Not as regular user, else we could not umount /home as it would be busy.
umount /home # This makes the previous /home accessible again.
rm -r /home/* # We delete the content of the previous /home but keep the directory.
mount /home # Now /dev/sdb1 is mounted as /home

Move only the largest files from /home to the hard disk

In the example above the /home directory has been completely transferred from the SSD to the hard disk.

It may be preferable to store only large files on the hard disk and to leave hidden directories and files on the SSD, which are subject to frequent writes, much faster on an SSD.

To do this, you can name the mount point of the /dev/sdb1 partition /data in /etc/fstab, for example, then once this partition is mounted, copy the directories to be transferred from /home to /data, and finally replace these directories in /home with symbolic links to the directories in /data. For instance, once the /data directory is created and mounted on /dev/sdb1:

chown -R didier:users /data
mv /home/didier/Images /data
ln -s /data/Images /home/didier/Images

This way of doing is to be adapted if the system is multi-user, for example by creating a subdirectory per user in /data.

Move /home to the hard disk but store frequently changed files on the SSD

On the other hand it is possible to selectively store frequently changed files on the SSD, like those contained in ~/.mozilla, ~/.thunderbird or ~/.purple.

For example, you can create a directory /data and a subdirectory /data/.thunderbird on the SSD, move ~/.thunderbird to it, and create a symbolic link /data/.thunderbird ⇒ ~/.thunderbird.

If btrfs is used for the core system it is recommended to create a subvolume for /data with the following commands, before moving ~/.mozilla or ~/.thunderbird there, like this:

mount /dev/sda3 /mnt subvolid=0
btrfs subvolume create /mnt/@data
mkdir /data
umount /mnt

Then insert a line in /etc/fstab to mount /data at system boot (<uuid1> is the UUID value of /dev/sda3):

UUID=<uuid1> /data btrfs subvol=/@data,discard=async,compress=zstd:3,noatime 0 0

Then reboot and type the following commands:

chown -R didier:users /data
mv /home/didier/.thunderbird /data
ln -s /data/.thunderbird ~/.thunderbird