High load with Backintime via cronjob

Asked by Christian Velten

Hi there,

I observe a quite high system load after starting BackInTime as a cronjob.

"High system load" = RAM/mem usage >80% (regularly), high CPU load >80% for 4 cycles (occasionally)

I found found an older report in another forum mentioning that this could be due to running BIT as cronjob (https://forums.opensuse.org/showthread.php/407913-Enormous-memory-usage-rsync-through-Back-In-Time).

What can I do? Any room for optimization? More an rsync issue?

Best regards
Christian

Question information

Language:
English Edit question
Status:
Solved
For:
Back In Time Edit question
Assignee:
No assignee Edit question
Solved by:
Christian Velten
Solved:
Last query:
Last reply:
Revision history for this message
Germar (germar) said :
#1

Did you disable 'nice' and 'ionice' for cronjobs in Expert Options? With both activated the cronjob should run silently in background without interrupting your daily work. RAM caching can also be reduced by activating 'nocache'.

Which process keeps the CPU busy? Is it rsync or python when you look into 'htop'

Revision history for this message
Christian Velten (velten-c) said :
#2

Der Germar,

Sorry for the delay. As those tasks are running at times I am usually not accessing the server, I had to wait for the weekend.

* 'nice' and 'ionice' were/are enabled

* I now activated the 'nocache' option to see if there is going to be a difference, keep you in the loop

* during the high system load period, rsync seems to be taking <40% CPU most of the time
* but python - runing in parallel - creates <80%(!) CPU load peaks, not all of the time but regularly at the high system load period, more to the end of the backup

Best
Chris

Revision history for this message
Germar (germar) said :
#3

While running the rsync process the python process only read all stdout/err lines coming from rsync and processes them (filter out progress and write log). This should not stress your CPU. After rsync is done (and if 'Full rsync mode' is deactivated) Python will create a permission list of all files. This can stress the CPU.

Please add some more information about your system:
- Processor / RAM
- HDD or SSD
- Operating System
- backup mode (local connected HDD or remote via SSH)
- BIT version

Revision history for this message
Christian Velten (velten-c) said :
#4

First, ...
your explantion fits to my observations. The high CPU load by python was definitely more to the end of the backup. Thank you for the background!

Second, ...
it looks like that the "nocache" option of BIT has done quite a good job. High system load notes have dramatically decreased since I switched in on. I keep an eye on it.

Third, ...
CPU AMD K8 Athlon64 3700+
4GB RAM
128kB L1/1MB L2
HDDs (SATA) only
Ubuntu 14.04LTS
local backup on HDD
BIT version 1.1.6

Best
Chris

Revision history for this message
Germar (germar) said :
#5

Activating 'Full rsync mode' will help even more and will speed up the whole process, too. Just make sure your destination drive has a native Linux filesystem.

'Full rsync mode' will not create the permission list because it stores all files with it's correct permissions.

That machine should definitely be powerful enough... ;-)

Revision history for this message
Christian Velten (velten-c) said :
#6

> That machine should definitely be powerful enough... ;-)

Also thought so. :-)

Thank your for the additional advice. I am going to check this out.

Before final notice, I would also like to watch it for a couple of days.

Thanks a lot
Chris

Revision history for this message
Christian Velten (velten-c) said :
#7

After watching the system load for a couple of days, I am happy to confirm that CPU load by rsync and Pythonhas been decreased by the measures suggested by Germar before.

From time to time, I still see an isolated quite high RAM load (>80%) at the time BIT is doing a backup. But I was not able to clearly assign it to a particular process so far. And overall factually high RAM load warnings have substantially decreased after applying the measures.

Thanks a lot again, Germar.

Best
Christian