How does BIT handle data integrity?

Hi Niklas,

BackInTime uses rsync to sync files between local and remote drive (remote in this case can be an external drive or a remote host, doesn't matter). By default (without --checksum) rsync compares files size and modification time to decide if a file has changed and need to be transferred.

If rsync decides the file need to be transferred (or if the file is missing) it will create a rolling checksum and a full checksum of the local file and one rolling checksum of the remote file. Based on both rolling checksums the remote site knows which blocks of the file has changed and will request those blocks from local site. Together with the unchanged blocks the remote site will reconstruct the file and compare it with the full checksum from the local file.

So rsync takes care of integrity of new and changed files. But it could happen that files get corrupted on the remote site due to disk failure or bit rot. Because such corruptions doesn't change file size or modification time rsync will not identify those files as changed and simply hard-link them with each and every new snapshot.

The only way to find those corruptions is to run rsync with the --checksum argument. This makes rsync to ignore size and mod-time and create rolling and full checksums for all files both local and remote to decide if a file needs to be transferred. This could easily slow down the process by factor 10.

So I'd suggest to run BackInTime only once a month (or once a week) with --checksum like I wrote in that FAQ

Regards,
Germar