Unexpected backup every half an hour
I run backintime backups on two usb drives, viz:
Main profie: One the hour (Schedule every hour through the optins)
Profile 2: Scheduled at 30 minutes past each hour, as detailed in https:/
However, a few weeks ago, without me having made any changes, Profile 2 is no longer backing up, and the Main profile is backing up every half an hour (it is still set to every hour). Here is the output of crontab -e
30 */1 * * * /usr/bin/nice -n 19 /usr/bin/ionice -c2 -n7 /usr/bin/backintime --profile-id 2 --backup-job >/dev/null 2>&1
#Back In Time system entry, this will be edited by the gui:
0 * * * * /usr/bin/nice -n 19 /usr/bin/ionice -c2 -n7 /usr/bin/backintime --backup-job >/dev/null 2>&1
Could anybody please advise me how to resolve this situtation?
With thanks, Martin
Question information
- Language:
- English Edit question
- Status:
- Solved
- Assignee:
- No assignee Edit question
- Solved by:
- Martin Fisher
- Solved:
- Last query:
- Last reply:
Revision history for this message
|
#1 |
Hi Martin!
please post the output of 'grep -A4 "CRON" /var/log/syslog | tail -n30'
Regards,
Germar
Revision history for this message
|
#2 |
Hi Germar! Many thanks, and please see below.
martin@martin:~$ grep -A4 "CRON" /var/log/syslog | tail -n30
Sep 17 10:00:01 martin backintime (martin/1): INFO: Lock
Sep 17 10:00:01 martin backintime (martin/1): INFO: Inhibit Suspend started. Reason: take snapshot
Sep 17 10:00:01 martin backintime (martin/1): INFO: Take a new snapshot. Profile: 1 Main profile
--
Sep 17 10:17:01 martin CRON[5875]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Sep 17 10:17:38 martin org.gnome.
Sep 17 10:17:40 martin gnome-session[
Sep 17 10:17:40 martin gnome-session[
Sep 17 10:17:40 martin gnome-session[
--
Sep 17 10:30:01 martin CRON[6012]: (martin) CMD (/usr/bin/nice -n 19 /usr/bin/ionice -c2 -n7 /usr/bin/backintime --profile-id 2 --backup-job >/dev/null 2>&1)
Sep 17 10:30:01 martin backintime (martin/1): INFO: Run command 'backup-job' instead of argument '--backup-job' due to backwards compatibility.
Sep 17 10:30:01 martin backintime (martin/1): INFO: Lock
Sep 17 10:30:01 martin backintime (martin/1): INFO: Inhibit Suspend started. Reason: take snapshot
Sep 17 10:30:01 martin backintime (martin/1): INFO: Take a new snapshot. Profile: 1 Main profile
--
Sep 17 11:00:01 martin CRON[6584]: (martin) CMD (/usr/bin/nice -n 19 /usr/bin/ionice -c2 -n7 /usr/bin/backintime --backup-job >/dev/null 2>&1)
Sep 17 11:00:01 martin backintime (martin/1): INFO: Run command 'backup-job' instead of argument '--backup-job' due to backwards compatibility.
Sep 17 11:00:01 martin backintime (martin/1): INFO: Lock
Sep 17 11:00:01 martin backintime (martin/1): INFO: Inhibit Suspend started. Reason: take snapshot
Sep 17 11:00:01 martin backintime (martin/1): INFO: Take a new snapshot. Profile: 1 Main profile
--
Sep 17 11:17:01 martin CRON[7047]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Sep 17 11:25:04 martin smartd[1003]: Device: /dev/sdb [SAT], 3 Currently unreadable (pending) sectors
Sep 17 11:25:04 martin smartd[1003]: Device: /dev/sdb [SAT], Failed SMART usage Attribute: 1 Raw_Read_
Sep 17 11:30:01 martin CRON[7152]: (martin) CMD (/usr/bin/nice -n 19 /usr/bin/ionice -c2 -n7 /usr/bin/backintime --profile-id 2 --backup-job >/dev/null 2>&1)
Sep 17 11:30:01 martin backintime (martin/1): INFO: Run command 'backup-job' instead of argument '--backup-job' due to backwards compatibility.
Sep 17 11:30:01 martin backintime (martin/1): INFO: Lock
Sep 17 11:30:01 martin backintime (martin/1): INFO: Inhibit Suspend started. Reason: take snapshot
Sep 17 11:30:01 martin backintime (martin/1): INFO: Take a new snapshot. Profile: 1 Main profile
martin@martin:~$
Revision history for this message
|
#3 |
Looks like a bug in current dev version. But I'm not yet able to reproduce/fix it.
You did not open Settings dialog in BIT and pressed OK after updating to current dev version. Please don't do this until we found the bug ;-) otherwise he might slip away as the problem might be solved with writing a new crontab from BIT.
Both writing of crontab and parsing the command line arguments have changed in current dev version.
Please run 'crontab -e' and add --debug after /usr/bin/
Sep 1x xx:30:01 martin CRON[6012]: (martin) CMD (/usr/bin/nice -n 19 /usr/bin/ionice -c2 -n7 /usr/bin/backintime --debug --profile-id 2 --backup-job >/dev/null 2>&1)
Also please run 'backintime --debug --profile-id 2 backup-job' in Terminal and post the output, too.
Many thanks for testing the dev version. I was about to release a new version this weekend. But this bug is a show-stopper.
Kind regards,
Germar
Revision history for this message
|
#4 |
Many thanks Germar. Herewith the details from /var/sys/log after the first half-hour backup (this is the complete BiT section):
Sep 18 09:30:01 martin CRON[4425]: (martin) CMD (/usr/bin/nice -n 19 /usr/bin/ionice -c2 -n7 /usr/bin/backintime --debug --profile-id 2 --backup-job >/dev/null 2>&1)
Sep 18 09:30:01 martin backintime (martin/1): DEBUG: [common/
Sep 18 09:30:01 martin backintime (martin/1): INFO: [common/
Sep 18 09:30:01 martin backintime (martin/1): DEBUG: [common/
Sep 18 09:30:01 martin backintime (martin/1): DEBUG: [common/
Sep 18 09:30:01 martin backintime (martin/1): DEBUG: [common/
Sep 18 09:30:01 martin backintime (martin/1): DEBUG: [common/
Sep 18 09:30:01 martin backintime (martin/1): DEBUG: [common/
Sep 18 09:30:01 martin backintime (martin/1): DEBUG: [common/
Sep 18 09:30:01 martin backintime (martin/1): DEBUG: [common/
Sep 18 09:30:01 martin backintime (martin/1): INFO: [common/
Sep 18 09:30:01 martin backintime (martin/1): INFO: [common/
Sep 18 09:30:01 martin backintime (martin/1): DEBUG: [common/mount.py:50 Mount.__init__] pw-cache is not running
Sep 18 09:30:01 martin backintime (martin/1): DEBUG: [common/mount.py:58 Mount.__init__] Call command: backintime pw-cache start
Sep 18 09:30:01 martin backintime (martin/1): INFO: [common/
When I run backintime --debug --profile-id 2 backup-job the output is very long (a lot of Permission denied and similar) and I cannot capture it all in the terminal. Could you pleas kindly send me an instruction for how to direct the output to a file, so that I can capture everything and send to you (unfortunately I don't know how to do this).
With thanks, Martin
Revision history for this message
|
#5 |
Please download this patch http://
sudo patch /usr/share/
This will add some more debug messages that may help to find the bug. After patching please post the above syslog part again. To write the output into file use
backintime --debug --profile-id 2 backup-job >~/bit.log 2>&1
Revision history for this message
|
#6 |
Many thanks Germar. Herewith the syslog (output file of backup to follow):
Sep 19 12:30:01 martin CRON[3622]: (martin) CMD (/usr/bin/nice -n 19 /usr/bin/ionice -c2 -n7 /usr/bin/backintime --debug --profile-id 2 --backup-job >/dev/null 2>&1)
Sep 19 12:30:02 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:02 martin backintime (martin/1): INFO: [common/
Sep 19 12:30:02 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:02 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:02 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:02 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:02 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:02 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:02 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:02 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:02 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:02 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:02 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:02 martin backintime (martin/1): INFO: [common/
Sep 19 12:30:02 martin backintime (martin/1): INFO: [common/
Sep 19 12:30:02 martin backintime (martin/1): DEBUG: [common/mount.py:50 Mount.__init__] pw-cache is not running
Sep 19 12:30:02 martin backintime (martin/1): DEBUG: [common/mount.py:58 Mount.__init__] Call command: backintime pw-cache start
Sep 19 12:30:02 martin backintime (martin/1): INFO: [common/
Sep 19 12:30:02 martin kernel: [ 5279.637531] EXT4-fs error: 30 callbacks suppressed
Sep 19 12:30:02 martin kernel: [ 5279.637535] EXT4-fs error (device sdc1): ext4_lookup:1417: inode #16386: comm python3: deleted inode referenced: 17828897
Sep 19 12:30:02 martin backintime (martin/1): INFO: [qt4systrayicon] begin loop
Sep 19 12:30:07 martin backintime (martin/1): INFO: [common/
Sep 19 12:30:07 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:07 martin backintime (martin/1): DEBUG: CONTINUE: --include=
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): INFO: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): WARNING: [common/
Sep 19 12:30:10 martin backintime (martin/1): INFO: [common/
Sep 19 12:30:10 martin kernel: [ 5288.164117] EXT4-fs error (device sdc1): ext4_lookup:1417: inode #16386: comm python3: deleted inode referenced: 17828897
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: CONTINUE: '20141120-
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin kernel: [ 5288.165726] EXT4-fs error (device sdc1): ext4_lookup:1417: inode #16386: comm python3: deleted inode referenced: 17828897
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: CONTINUE: '20141031-
Sep 19 12:30:10 martin backintime (martin/1): INFO: [common/
Sep 19 12:30:10 martin backintime (martin/1): DEBUG: [common/
Revision history for this message
|
#7 |
...and using backintime --debug --profile-id 2 backup-job >~/bit.log 2>&1 here is the backup log. It is HUGE so I am pasting here just the first few and last few lines (let me know if you require more).
beginning:
find: ‘/media/
find: ‘/media/
find: ‘/media/
find: ‘/media/
find: ‘/media/
find: ‘/media/
find: ‘/media/
find: ‘/media/
find: ‘/media/
find: ‘/media/
find: ‘/media/
find: ‘/media/
find: ‘/media/
find: ‘/media/
find: ‘/media/
find: ‘/media/
find: ‘/media/
find: ‘/media/
find: ‘/media/
find: ‘/media/
find: ‘/media/
find: ‘/media/
end:
rm: cannot remove ‘/media/
rm: cannot remove ‘/media/
find: ‘/media/
find: ‘/media/
find: ‘/media/
find: ‘/media/
find: ‘/media/
find: ‘/media/
find: ‘/media/
rm: cannot remove ‘/media/
rm: cannot remove ‘/media/
rm: cannot remove ‘/media/
rm: cannot remove ‘/media/
rm: cannot remove ‘/media/
rm: cannot remove ‘/media/
rm: cannot remove ‘/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
WARNING: Command "find "/media/
WARNING: Command "rm -rf "/media/
Back In Time
Version: 1.1.8~beta2
Back In Time comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under certain conditions; type `backintime --license' for details.
INFO: Lock
INFO: Inhibit Suspend started. Reason: take snapshot
INFO: Take a new snapshot. Profile: 1 Main profile
INFO: Compare with old snapshot: 20150919-120002-800
INFO: Create hard-links
INFO: Call rsync to take the snapshot
INFO: Save config file
INFO: Save permissions
INFO: Create info file
INFO: Remove backups older than: 20050901-000000
INFO: [smart remove] remove snapshots: ['20150317-
INFO: Keep min free disk space: 1024 Mb
INFO: Keep min 2% free inodes
INFO: Unlock
INFO: Release inhibit Suspend
INFO: [qt4systrayicon] begin loop
INFO: [qt4systrayicon] end loop
Revision history for this message
|
#8 |
Hi Martin,
Good news first: it will be fixed as soon as you open and close Settings dialog once. But I'd highly appreciate if you'd stay with me until I found the root of the problem.
The problem is in these three lines:
INFO: [common/
DEBUG: [common/
DEBUG: [common/
The aliasParser doesn't get get --debug and --profile-id arguments. Last line should be instead:
DEBUG: [common/
Which Ubuntu version do you use? Please post the output of
python3 --version
python3 -c 'import argparse; print(argparse.
-------
There is an other problem with smart-remove on your drive. Looks like it isn't able to remove couple old snapshots. But I'd like to fix the profile issue first and comeback to the smart-remove issue later on. So please deactivate smart-remove for now with:
sed -e 's/\(profile1.
Revision history for this message
|
#9 |
Hi Germar - Thanks again and herewith:
Ubuntu 15.04
python3 --version
Python 3.4.3
python3 -c 'import argparse; print(argparse.
1.1
I have deativted smart-remove
Martin
Revision history for this message
|
#10 |
Okay, I was now able to reproduce this in a 15.04 VM. Seems to be related to 15.04 as it works well on 14.04
Thanks a lot for your help. Feel free to open and close Settings dialog in BIT to fix that problem for now. I'll come back later to help you fixing the smart-remove issue
Revision history for this message
|
#11 |
Hi Germar
Many thanks, and I have done that. However, I am still unable to run a Profile 2 backup, and Profile 1 still makes a backup on the half hour as well as on the hour.
Martin
Revision history for this message
|
#12 |
Oh, indeed. So I'm searching at the wrong place m)
Revision history for this message
|
#13 |
1.1.8~beta3 on testing PPA will fix the issue
Revision history for this message
|
#14 |
Many thanks Germar. I have tested the update and now both of my profiles run. Shortly Profile 2 will run on the half hour and I will report whether this backups correctly.
Revision history for this message
|
#15 |
...and profile 2 now backups up on the half hour, and within profile 2 (not within profile 1, as formerly).
Revision history for this message
|
#16 |
Thanks for the feedback. So lets activate smart-remove again (you can do that in Settings) and work on that one ;-)
First of all please make sure the drive is healthy. Run 'df /media/
If this doesn't help please run:
ls -la /media/
and post the output.
Revision history for this message
|
#17 |
Hi Germar
Many thanks. I ran fsck and there were many errors on the drive, and I let fsck correct them all. I have now run profile 1, activated smart-remove and run a backup. A number of older backups have now being removed. I keep one snapshot per month for the last 24 months but some months had several: now only March 2013 has an extra backup.
However, I do not think the problem is yet completely resolved. As I retain one snapshot per year for all years, I think the one snapshot per month of 2013 should now have been removed and only a single snapshot retained. I ran backintime with debug (your earlier suggestion backintime --debug --profile-id 2 backup-job >~/bit.log 2>&1) and it seems there are a lot of permission denied problems associated with some files in the 2013 backups.
I could probably quite safely delete all 2013 snapshot except the December 2013. I searched in bit.log from the above command an none of the errors are associated with the December 2013 backup, which is 20131220). Would you advise doing this or would it be best to otherwise resolve the problem?
Again, I am very grateful for your expert advice.
Yours, Martin
Revision history for this message
|
#18 |
I guess that snapshots (or parts of it) is owned by an other user or root. You can check this with (one line):
find /media/
I'd just remove them manually with:
rm -rf /media/
Or if that still doesn't work use:
sudo rm -rf /media/
As your drive had many problems I suggest running one snapshot with 'backintime --checksum backup' to make sure all files are the healty. This will use files checksums instead of time and size to check if a file has changed. But it will take a lot more time.
Revision history for this message
|
#19 |
Thank you again Germar. I have now deleted all of the 2013 backups except for that of December (I checked and some of them had files owned by root rather than by myself).
I tried to run backintime --checksum backup' but BiT complains about --checksum. I checked man backintime and I can't understand why this is. I am now running this backup using the appropriate option in the GUI (it is taking a loooong time however!).
To improve my meagre commend line knowledge I have tried to understand
find /media/
I am guessing that '! - user martin' means not belonging to the user martin?
And that 'exec ls -lad {} +' means execute ls to list files (but {} + is a mystery to me).
With thanks, Martin
Revision history for this message
|
#20 |
Hmmm. The new argument parser only takes 'backintime backup --checksum' and not the other way around >:|
I'll have a look if I can change this.
'-user martin' is a test command for find. It will search for files which belongs to 'martin'. With '!' you can invert every test in find. So '! -user martin' will search for all files which doesn't belong to martin.
If you would run 'find /path ! -user martin' by it's own it would have shown you all files that are not owned by martin. But you wouldn't know to whom they belong. 'find' as no options to output files in a long list format. So I used a trick:
'-exec COMMAND {} +' is an action for find. It will execute COMMAND with all files it found until the new command reaches the maximum argument lenght. {} will be replaced by the filenames. There is an other method with '-exec COMMAND {} \;' which will execute COMMAND for every file. The first one is of course faster.
So '-exec ls -lad {} +' will construct 'ls -lad file1 file2 file3 file4...' and once that command-line reached maximum argument length it will execute it and start building a new command 'ls -lad file624 file625 file626...'
Revision history for this message
|
#21 |
Hi Germar
Many thanks for explaining the commands - most appreciated.
Martin
Revision history for this message
|
#22 |
Just as comment:
The find command does have an option to output files in a long list format, namely the -ls action, so the command
find /media/
might have worked as well.
Revision history for this message
|
#23 |
You're right Manfred. Didn't know that one :)
Thanks