Possible to use lsyncd or similar to trigger backintime snapshot?

All you have to do is run "backintime --backup" (with or without nice / ionice, ...).
If you can do it from lsyncd then you have all you need.

Reagrds,
Dan

Thanks Dan, yes, that would trigger a complete snapshot. However, I would prefer only that particular file that changed or newly arrived to be backed up, not to let backintime do a scan of the entire file system (that would defeat the purpose....).

BIT process is:

If the last snapshot exists:
1. compare with the last snapshot
2. duplicate the last snapshot using hard-links

For all snapshots:
3. call rsync to backup

What can be done is add an option to skip step 1 and do a partial step 3 (call rsync only for changed path / file). Step 2 can't be changed. If this looks fine for you open a bug for it.

Regards,
Dan

My understanding is that the file comparisons that BiT performs by default (without "use checksums to detect changes" enabled) are fairy light-weight, in that it depends on timestamps and/or file sizes being different between the current file and the prior snapshot.

While walking the directory tree is less intense than reading every byte to calculate and compare checksums, it still can generate a considerable disk and CPU load.

The technology underlying lsyncd (http://code.google.com/p/lsyncd/ a backup/sync tool) is inotify or fsnotify (http://lwn.net/Articles/318618/), which lets applications subscribe to receive notifications when an inode changes.

At a high level, your application subscribes to receive the change notifications, and would need to log them, weeding out duplicates. Then when it is time to create a backup, it generates a file list from this log, thus avoiding a recursive directory comarison.

I think there are two ways to accomplish integration with BiT:

1. Have BiT (specifically a new daemon component) use the fsnotify API directly, which would build a database of files to be backed up at the next snapshot time.

2. Create a small helper program that is invoked by lsyncd and writes files to a database (SQLite?), then modify BiT to pull its list of files to rsync from the database.

#2 would probably be faster to prototype, at the expense of being dependent on another tool.

Given that BiT starts off by generating a full tree of hard links for each snapshot, half of the recursive directory tree walking will be unavoidable. But it could substantially reduce the processing required for the rsync step, if you have a high number of files and only a few change between snapshots.

 -Tom

FYI, a Python application that uses inotify:
http://objectmix.com/python/249531-realtime-fs-mirror-application-backup-python-linux-inotify.html

other related projects:
http://inotify.aiken.cz/?section=incron&page=about&lang=en
http://code.google.com/p/sersync/

and a discussion thread on the topic of backup tools that use inotify:
http://serverfault.com/questions/7969/is-there-a-working-linux-backup-solution-that-uses-inotify