Back In Time

After restore BIT automatically starts a snapshot on files restored

Asked by Sven Eriksson

Dear Germar/Dan,

BIT totally saved me today when somehow a directory with 3+ years of work suddenly was gone.
Many thanks for this software.

Restoring took a good 6 hours. However, after that something strange happened:

After restoring, BIT started to immediately make a snapshot on very many files
in the restored directory that were not changed in a long time (but were just restored).

It is still restoring as I write this.

What is going on? Is BIT messing up my files now?

Could it be that BIT concurrently while it was doing the restoring also made a snapshot on the incomplete
folder structure and when everything was restored, BIT found all these "new" (restored file) compared to the
the possible snapshot during restore?

Any other explantions?

Thanks for any insight into this problem.

(I noticed during the end of the restore, when BIT fixes the file permissions, that it tried to set permissions for /home
but failed (presumably because it runs as me/user) and then it finnished. Any adverse implications from that
related to this problem with the seemingly irrational snapshot after restore?)

Best regards,

Sven

Question information

Language:
English Edit question
Status:
Solved
For:
Back In Time Edit question
Assignee:
No assignee Edit question
Solved by:
Germar
Solved:
Last query:
Last reply:
Revision history for this message
Germar (germar) said : 		#1

Hi Sven,
I'm glad to hear that restore worked for you.

Are you using mode SSH with 'Full rsync mode'? With SSH and without 'Full rsync mode' permissions are stored in an extra text file and will be restored after everything is done.

In this case probably permissions wasn't restored when the new snapshot was taken and rsync changed permissions in new snapshot like they where at that moment on your local machine. Take a look into the last snapshot log if you see something like '<f.p....... path/to/file'. This means that a file (f) was transfered to remote host (<) because of changes in permissions (p). If the file was changed you would see t for time difference and/or s for size or maybe c for checksum if you run with --checksum.

I'd recommend to delete the new snapshot. Your other snapshots were not touched at all. So when restore is done you should be able to create a new snapshot where nothing has changed.

Failing while restore permissions for /home is just normal as your user don't have permissions to change these. But it wouldn't change anything, just reset the same permissions. So you don't need to worry about that.

Regards,
Germar

Revision history for this message
Sven Eriksson (eriksson-sven3) said : 		#2

Hi Germar,

Thanks for your answer.

Yes I use SSH with "Full rsync mode".

I am not sure that I follow what you wrote, from my syslog I can see
that I started restoring at 10.34:

Jul 12 10:34:30 host backintime (user): INFO: Restore: /home/user/dir

The restore was concluded at ~16-1700h

However, something is really strange, in the GUI I can see a snapshot from 11:00:01 and 12:00:01
So it seems it made a snapshot while it was also restoring?

In the log for the 11:00 snapshot there are a lot of rows like this:
[I] Make snapshot [Swedish: ta ögonblicksbild] (rsync: BACKINTIME: cd..t...... home/user/dir /dir/dir/)

No row with '<f.p....... path/to/file'.

In the log for the 12:00 snapshot (restore not concluded yet, but still this snapshot) there are
very many rows like this:
[C] <f+++++++++ home/user/dir/dir/file

So, looks like it is restoring files here somehow? Please note that this file [C] <f+...home/user/dir/dir/file was restored between 10:34 and 12:00 since it was in a directory that prior to 10:34 was completely deleted. It looks like it is being written back to the remote from which the restore was done. I don't get it.

By the way, when I look at the logs it says at the end of them: saving permissions
Seems like it uses the permissions file even though I have Full rsync mode checked.

What has happend? Should I be worried that my files are messed up? What can I do?

I was worried that I had got some rootkit that removed my directory, but after further inspection it looks
like it must have been me who did it (don't know how it could happen).
Anyway, I will now set up immutable bit with chattr for the uppermost (root) directory of the backup directory
chain. (In the root directory there are only further directories and I am not going to add or subtract directories there.)
Also am considering changing all permissions for all files in the directory to be only readable by me.
Is any of this going to force BIT to take a new snapshot of all files?

Greatful for any guidance.

Regards,

Sven

Revision history for this message
Best Germar (germar) said : 		#3

You don't need to worry. Your files should be fine. New snapshots don't change the older snapshots (as long as they are not deleted by smart remove).

What had happen in your case is when the 11:00:01 snapshot was taken just a few files had already been restored but all folders where created locally by rsync with current time (rsync will correct the folders time as soon as it will restore the files inside that folders). So in the new snapshot most files where deleted (those which where not restored yet) and times of remote folders where changed to match the (wrong) time at your local machine (that's 'cd..t......' (C)ange the (D)irectory because of (T)ime differences).

While running 12:00:01 snapshot couple more files where already restored. So the new snapshot added the 'new' files (<f+++++++++). They are not incremental anymore.

But again, this is nothing to worry about. Just delete those snapshots that where made while restoring and everything should be fine again.

Changing permissions for your local files will not force BIT create new instances of those files. I'm not even sure if the immutable bit is recognized by rsync and the 'save permissions' process. But BIT will fail to restore permissions for those folders like it failed for your /home which is totally okay.

'saving permissions' does run because you are using mode SSH. Your remote user might not be able to create files with the same permissions/ownership as your local user and it would be a very bad idea to use remote root for this. So I forced to store permissions into txt file with mode SSH even if 'Full rsync mode' is checked.

I'll make BIT to suspend new snapshots while restore is running. This should reduce confusion next time.

Regards,
Germar

Revision history for this message
Sven Eriksson (eriksson-sven3) said : 		#4

Thanks Germar, that solved my question.

Revision history for this message
Sven Eriksson (eriksson-sven3) said : 		#5

Germar,

Thank you very much for your patient replies, I'll mark my question/problem as solved.

As an aside, do you take pull requests?

If so, do you have some very minor and not very important bug/add feature way down the bug/feature wish list that
I could look at/work on over the summer? Would be kind of fun to acquaint my self with the code base.

If there are unittests for the code, I could start looking there to get to know it a little.

Anyway, again, thank you for providing this excellent piece of software.

Kind regards,

Sven

Revision history for this message
Germar (germar) said : 		#6

Hi Sven,

Yes sure! That would be great!

There are no tests yet. I only run BIT in some VM's and test manually. Not very professional but I'm no coder at all. I just do this for fun, too.

You could start with the problem you had in here if you like. Add a lock for restore and check the lock before a new snapshot will run. Locks can be done with common/applicationinstance.py. You just need to add a new lock file in common/config.py and set/unset/check the lock in common/snapshot.py

Or take a look at bug #1340266 or bug #823719. You can also contact me per email on 'germar DOT reitze HOSTED_ON gmail DOT com' or jabber on 'germar HOSTED_ON jabber DOT ccc DOT de'

Kind regards,
Germar