What is ZFS?
Author: Domeny.ForSale editorial team Published
In brief: ZFS is a file system combined with a mechanism for managing disk space. It verifies data integrity, supports snapshots and lets you combine storage devices into storage pools. It is particularly useful for servers and NAS devices, which share files over a network, but it does not replace a backup.
How ZFS works and how it differs from a conventional file system
A file system determines how data is stored and located on a storage device. ZFS also handles functions that other solutions often assign to a separate layer: combining disks, providing data redundancy and allocating space. This gives it direct visibility into both the file structure and where the files are stored.
ZFS originated at Sun Microsystems. Today, many Linux and FreeBSD-based solutions use OpenZFS — an implementation of ZFS developed as open-source software. Feature availability and installation methods depend on the operating system and version, so configuration instructions must be chosen for the specific environment.
Storage in ZFS has several levels:
- Pool (zpool) is shared storage space built from one or more groups of devices.
- Vdev is a logical device that makes up a pool, such as a single disk, a mirror or a RAID-Z group.
- Dataset is a separate file system within a pool that can have its own settings, such as compression and a space limit.
You can therefore create separate datasets for documents, media and computer backups without rigidly dividing disks into fixed-size partitions. Datasets share the available free space, subject to configured limits and reservations.
How ZFS detects data corruption
ZFS uses checksums: values calculated from the contents of data. During reads, it checks whether the stored blocks match their checksums. This makes it possible to detect issues such as silent corruption, where a device returns incorrect content without reporting a read error.
Detecting an error does not always mean it can be repaired. If the pool has suitable redundancy, such as a mirror, ZFS can retrieve a valid copy of a block and repair the corrupted one. On a single disk, it can detect corruption in a file’s data, but it cannot reconstruct its contents without an additional valid copy.
ZFS also uses copy-on-write: rather than overwriting a block that is in use, it writes the changed data to a new location and then updates the references. This helps maintain a consistent file system structure and forms the basis of snapshots. However, it does not mean that every application’s latest changes will necessarily be preserved after a sudden power loss.
A scrub provides periodic checking of stored data against its checksums. When redundancy is available, it also allows detected errors to be repaired. A scrub complements disk monitoring, but it does not replace disk diagnostics or backups.
Snapshots, compression and backups
A snapshot preserves the state of a dataset at a chosen point in time. Initially, it takes up little additional space; its space usage grows as old blocks of changed or deleted data need to be retained. This is why deleting a large file will not always free up space if a snapshot still retains its contents.
For example, you create a snapshot before organising your family photos. After accidentally deleting a directory, you can recover the earlier version without restoring the entire collection from an external disk. However, a snapshot kept in the same pool will not help if all storage devices are lost, the device is stolen or an attacker with sufficient permissions deletes the snapshot.
ZFS can send snapshots to another pool, including incrementally, by transferring changes relative to an earlier state. This forms the basis of convenient replication. However, the safety of such a copy depends on its storage location, permissions and policies for retaining older versions.
Compression performed during writes is also useful. It can save space and reduce the number of disk operations, but the results depend on the data: text usually compresses better than finished videos or archives. It should not be confused with deduplication, which eliminates duplicate blocks but can require substantial memory resources and careful planning.
Mirror or RAID-Z: choosing a disk layout
A mirror stores the same data on several disks. With two drives of equal capacity, usable space is approximately equivalent to one of them, before accounting for overheads and the recommended amount of spare free space. This layout can continue operating after one disk fails.
RAID-Z distributes data and the information needed to reconstruct it across drives. RAID-Z1, RAID-Z2 and RAID-Z3 tolerate the failure of one, two or three disks respectively within a given vdev. The choice depends on the required capacity, workload and acceptable risk.
The layout of the entire pool is crucial: losing a vdev that stores data can mean losing the pool. Adding a single disk as another vdev to a protected mirror does not extend the mirror’s protection to that disk. Before expanding, also check what your version of OpenZFS supports — not every layout change is available or easily reversible.
When to choose ZFS
ZFS is a good candidate for storing documents, archives and backups when you need integrity checking and snapshots. However, it requires planning the disk layout, monitoring pool health and keeping spare free space. Performance and RAM requirements depend on the workload and enabled features; there is no single universal ratio of memory to disk capacity.
ECC memory, which detects and corrects certain errors in RAM, is beneficial in a server, but it is not an absolute requirement for running ZFS. It is best to give ZFS direct access to disks rather than hide them behind a hardware RAID array.
Before moving important data, draw up a vdev plan, decide where to keep an independent backup and test restoring a single directory. This test will tell you more about the readiness of your setup than simply creating a pool successfully.
See also: What is Linux?
Read also
