What is full rsync mode?

'Full rsync mode' uses 'rsync --link-dest' to create hard-links instead of running 'rsync --dry-run' then 'cp -aRl' and finally 'rsync'. This is a lot faster. The downside of this is, that your destination filesystem must support all linux attributes like permissions, user, group and time. I'd recommend using ext3 or 4. Also the snapshots are not 'read-only' any more with this mode because the permissions are not stored in a separate file but with the normal file permissions. So you could accidentally remove the snapshots.

If you're using a Win/Samba share or an (external) NTFS drive as destination you shouldn't use this. But with all other setups this should be fine and stunning fast.

Regards,
Germar

Germar, many thanks for the information on "full rsync mode".

Sorry to bother again, I have a small problem though, I may have messed things up in connection with full rsync mode. What can I do?

Before I got the information on full rsync mode I had already started an over night backup of 1.4G of data containing ~3000 files without activating full rsync mode.

In the morning it looked like the backup was almost ready, however in the status bar at the bottom of the window it said: "saving rights...". (this is actually a translation from the Swedish version I am using, not sure if this is exactly what is shown in the english version).

After a few hours nothing had happend, so I decided to try to abort the backup and do it with full rsync mode I now had read about. However, after trying to turn BIT off and on and after that uninstalling and reinstalling it, it still said "saving rights...". Finally I found a folder .local containing BIT data. I wiped that, did a hard uninstall and then reinstalled BIT.

Now, info about my server was somehow still there but at least BIT was not locked up anymore displaying "saving rights...". So I happily edited the main profile to do "full rsync mode" and started a new snapshot.

After some hours when I looked at the program, the transfer of files was ready but again BIT stated "Saving rights...". This state has prevailed for more than 8 hours now. It really looks like it does nothing/that BIT is somehow stuck. Also, since full rsync mode was indicated, I thought that BIT shouldn't try to save any rights, since they should already have been taken care of by the file system of the remote server?

What is going on? What can I do to get BIT unstuck and to a successful backup?

I am using the 1.26 version of BIT; my remote server is running ubuntu on top of ext4. Public/Private key SSH login is working perfectly, I can log in etc.

Any info on a solution to my problem would be much appreciated.

Regards,
/Sven

Hi Sven,

Can you please post the output of 'grep backintime /var/log/syslog'?
I reactivated the creation of the permissions file with SSH and 'Full rsync mode' because a normal (non root) remote user is not able to create files under other usernames than his own.
It really looks like BIT is crashing for whatever reason during creating of this permissions file. If the status message stuck even if you killed all running backintime processes than this is only because the last message is still present in '/home/<USER>/.local/share/backintime/worker.message'. In that case you can just delete that file and the message will disappear.

Regards,
Germar

Hi Germar,

Thank you for your feedback.

Here is the grep:

Sep 15 09:33:08 server backintime (user): INFO: [Password_Cache.Daemon.daemonize] start
Sep 15 09:33:08 server backintime (user): INFO: [Password_Cache.Daemon.daemonize] write pidfile
Sep 15 09:33:08 server backintime (user): INFO: [Password_Cache.run] start loop
Sep 15 11:35:21 server backintime (user): INFO: [Password_Cache.Daemon.daemonize] start
Sep 15 11:35:21 server backintime (user): INFO: [Password_Cache.Daemon.daemonize] write pidfile
Sep 15 11:35:21 server backintime (user): INFO: [Password_Cache.run] start loop
Sep 16 08:55:09 server backintime (user): INFO: [Password_Cache.Daemon.daemonize] start
Sep 16 08:55:09 server backintime (user): INFO: [Password_Cache.Daemon.daemonize] write pidfile
Sep 16 08:55:09 server backintime (user): INFO: [Password_Cache.run] start loop

- Permissions file: does the reactivation mean that there is not a big savings anymore in backup time when using "full rsync mode" compared to ordinary backup mode?

- worker.message: I deleted the file and the message is gone, but there was never a snapshot created in the sense that I can not see a snapshot time entry in the snapshot frame in BIT.

- One strange thing, I deleted earlier the whole .local BIT directory, did a purge/remove all files of the BIT package and the reinstalled BIT. In spite of this, when I then started BIT there was still information about the back up scheme I previously had (address of server, user name etc.). Thus, looks like info is stored elsewhere as well. Could there be some configuration problem at this other place that is corrupting things?

Any ideas? I am really keen on getting this setup working, since BIT seems to be such a nice alternative to Déjà Dup.
Hopefully we can find the problem.

Thanks for helping.

Regards /Sven

Your syslog didn't contain the backup process any more. Can you please either search for the latest snapshot in /var/log/syslog.1 and /var/log/syslog.2.gz, ...3.gz and so on (use 'zgrep' for the compressed files) or even better run backintime-gnome from Terminal, create a new snapshot and post the output here. '/home/<USER>/.local/share/backintime/takesnapshot_.log' can be useful, too. As this both files can become quite long you can also send those files per email to germar DOT reitze HOSTED_ON gmail DOT com.

The permission file should normally run quite fast (on my main computer the creation of the permission file takes 30sec of overall 8min backup time which took ~22min without 'Full rsync mode'). So even with permission file 'Full rsync mode' is still stunning fast :-)

The snapshot will be created in a 'new_snapshot' folder which will be renamed if the snapshot was successful. Because in your case BIT crashed this didn't happen and so you'll not see the new snapshot. You can select 'Keep snapshots on error' in Options so you don't need to transfer all the files again if it fails.

The config is stored in '/home/<USER>/.config/backintime/config'. As this file is per user it will not be removed even if you purge the installation.

I don't think there is a problem with you config. I would guess there is a special file in your backup folders that causes your problems. But I'm pretty sure that we will find the problem when you send me those files above.

Regards,
Germar

Germar,

Again, thanks for the feedback.

Sorry a bit late reply, I have been traveling on work duty.

I did the new snapshot with output to terminal, I will send it to you by e-mail.
I also did the extended greping and concatenated it into a file that I will send to you as well.

It occured to me that a filtering option that I set may perhaps be a problem?
I set BIT up to exclude pattern #*# in the hope that live backup emacs files that looks like #file.txt# would be excluded.
Not sure if this is ok.

Sounds great that BIT should be really fast also with the permission file!

I had the "keep snapshots on error" marked, but when I did the new snapshot it looks like BIT did it from scratch. Not sure why.

Many thanks for your very kind help with this.

Regards,

/Sven

Hi Germar,

Thanks!

I added "ServerAliveInterval 300" to /etc/ssh/ssh_conf and that seemed to do the trick!

Over night I started a new backup and it came through perfectly.

Many thanks for all the help. Really looking forward to using this software now.

Kind regards,

/Sven

Thanks Germar, that solved my question.

Great! I'll add this to BIT, too. So it doesn't need to be set in ssh_config anymore.

Good idea to put it in BIT itself!

Another small idea connected to ssh and BIT: before I started to use BIT I had turned off icmp/ping on my server, just to obfuscate it a little.

However, there is something in BIT that is relying on ping, so I had to turn it on. Perhaps it can be done in another way, without ping?

It does not seem very important; I was just thinking that if you are anyway looking at the ssh code in BIT, perhaps this would be a simple adjustment also in connection to that code?

Btw, BIT started to help me today by backing up my amended files every 1/2 hour while, as far as I can tell, consuming minimal resources. Great!

I just changed the ping to a port-check 3 days ago :D
Look at bug #1226718