Interested in an alternative engine in dejadup?

Asked by dstromberg

I've put together a backup engine, analogous to rsync --link-dest, in Python. It's called Backshift.

I've been looking for a backup scheduler system/GUI that uses rsync, that might be interested in adding support for backshift as an alternative to rsync.

Is Dejadup perhaps such a project?

Here's a URL to the project:
http://stromberg.dnsalias.org/~dstromberg/backshift/

The next feature I'm planning to add, is the ability to do backups without messing up a system's buffer cache - since almost by definition, backups read a large number of files once, and then don't read them again anytime soon.

Right now, backshift has:
1) Hybrid fullsaves/incrementals, like rsync --link-dest
2) The ability to chop files up into variable-length, content-based blocks; this is more effective than fixed-width blocking, though slower
3) The ability to deduplicate those blocks without requiring untoward amounts of VM
4) The ability to compress those blocks with xz (via ctypes, subprocess or os.popen) or bzip2
5) The ability to resume interrupted backups
6) The ability to expire old blocks and old metadata

It runs on CPython 2.x, CPython 3.x, PyPy and Jython, is thoroughly documented, is thoroughly pylint'd, and has a comprehensive automated test suite. It's also been tested on a very large number of operating systems, including many *ix's, OS/X and Windows 7 with Cygwin.

Files are selected using find (I've been considering adding a tar-like way of saying "Do everything in this directory, recursively" though), and restored by automatically assembling a tar archive from the compressed blocks, on the fly. This maximizes the use of other popular tools like find and tar, and allows one to restore via the network using just ssh and tar.

?

Question information

Language:
English Edit question
Status:
Answered
For:
Déjà Dup Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Michael Terry (mterry) said :
#1

See old bug 306992, which requests adding the ability for deja-dup to support multiple backends. I've got enough on my plate right now, but I'm willing to help someone that was interested in working on it.

Revision history for this message
Van Nilla (vanilla-z) said :
#2

Have you looked at DAR as an alternative to TAR? DAR supposedly can do record-correction with external partity-stripers as well. If you've ever had to deal with a corrupt tar archive, you'll appreciate DAR (Disk ARchive).

Can you help with this problem?

Provide an answer of your own, or ask dstromberg for more information if necessary.

To post a message you must log in.