Rdiff-backup directory structure

rdiff-backup enables simple and space-efficient backups under Linux. In the Using rdiff-backup under Linux article, we show how to use rdiff-backup. In this article, we explain which data is contained in the rdiff-backup-data directory and which information can be requested.

rdiff-backup-data

rdiff-backup-data directory structure on backup server.

rdiff-backup creates a 1:1 copy of the current data during a backup. Information on older backup-versions is stored in the rdiff-backup-data sub-directory.

This directory contains the following files and directories:

• access_control_lists.TIME.snapshot
• backup.log
• chars_to_quote
• current_mirror.TIME.data
• error_log.TIME.data
• extended_attributes.TIME.snapshot
• file_statistics.TIME.data.gz
• increments (directory)
• increments.TIME.dir
• mirror_metadata.TIME.snapshot.gz (The most recent TIME entry is always saved as .snapshot.gz, with a maximum of 8 older entries saved as .diff.gz to one .snapshot.gz.)
• restore.log (only after restores have been performed)
• session_statistics.TIME.data

backup.log

If an rdiff-backup run is interrupted or certain files could not be backed up, this is logged in the backup.log file.

SpecialFileError dev/log Socket error: AF_UNIX path too long
Previous backup seems to have failed, regressing destination now.
Previous backup seems to have failed, regressing destination now.

current_mirror.TIME.data

This file stores the PID of the rdiff-backup command that performs or performed the last backup run on this directory. At the same time, this file specifies the date of the last backup.

 PID 26233

file_statistics

Example of a backup run statistics file (the zipped file can be easily opened with vi, as vi supports the display of compressed data):

# Format of each line in file statistics file:
# Filename Changed SourceSize MirrorSize IncrementSize
. 0 0 0 NA
bin 0 0 0 NA
bin/bash 0 812212 812212 NA
[...]
var/log/mail.log 1 281141 1304109 114879
[...]

Increments

/rdiff-backup-data/increments$ ls -l
total 32
drwx------ 2 wfischer wfischer 4096 Sep 14 15:28 dir1
-rwxrwxr-x 1 wfischer wfischer    0 Sep 14 15:27 dir1.2015-09-14T15:27:37+02:00.dir
-rw-rw-r-- 1 wfischer wfischer   80 Sep 14 15:27 file1.txt.2015-09-14T15:27:37+02:00.snapshot.gz
-rw------- 1 wfischer wfischer    0 Sep 14 15:28 file2.txt.2015-09-14T15:27:37+02:00.missing

mirror_metadata

The compressed meta data can be also displayed by vi:

File .
  Type dir
  ModTime 1425809355
  Uid 0
  Uname root
  Gid 0
  Gname root
  Permissions 493
File bin
  Type dir
  ModTime 1418727096
  Uid 0
  [...]

restore.log

When restoring data, rdiff-backup logs this in restore.log:

$ cat restore.log 
Starting restore of mail.log.2015-02-01T00:01:18+01:00.diff.gz to /tmp/mail.log as it was as of Sun Feb  1 00:01:18 2015.

session_statistics

The following example shows the content of a session statistic:

$ cat session_statistics.2015-09-02T00:01:17+02:00.data
StartTime 1441144877.00 (Wed Sep  2 00:01:17 2015)
EndTime 1441147057.65 (Wed Sep  2 00:37:37 2015)
ElapsedTime 2180.65 (36 minutes 20.65 seconds)
SourceFiles 225971
SourceFileSize 21649997461 (20.2 GB)
MirrorFiles 224656
MirrorFileSize 22111665788 (20.6 GB)
NewFiles 1625
NewFileSize 266819836 (254 MB)
DeletedFiles 310
DeletedFileSize 43276628 (41.3 MB)
ChangedFiles 3335
ChangedSourceSize 3405919275 (3.17 GB)
ChangedMirrorSize 4091130810 (3.81 GB)
IncrementFiles 5275
IncrementFileSize 798350336 (761 MB)
TotalDestinationSizeChange 336682009 (321 MB)
Errors 0

More information on the analysis of this information is shown in the following article: Analysis of rdiff-backup statistics

Author: Werner Fischer Werner Fischer, working in the Knowledge Transfer team at Thomas-Krenn, completed his studies of Computer and Media Security at FH Hagenberg in Austria. He is a regular speaker at many conferences like LinuxTag, OSMC, OSDC, LinuxCon, and author for various IT magazines. In his spare time he enjoys playing the piano and training for a good result at the annual Linz marathon relay.

Translator: Alina Ranzinger Alina has been working at Thomas-Krenn.AG since 2024. After her training as multilingual business assistant, she got her job as assistant of the Product Management and is responsible for the translation of texts and for the organisation of the department.