object-auditor process consume disk I/O

Asked by mhy on 2011-12-07

hello! my swift is 1 proxy node + 5 zones(one storage node each zone, each storage node has 2T disk space), siwft1.4.3

on my storage node, the disk IO speed I got with the help of "time dd if=/dev/zero of=/srv/node/sda3/test bs=1M count=2000" is about 120MB/s;
But, once the object-auditor process of storage nodes is on, the IO speed will decrease violently, because the object-auditor has been always reading disk and generating read IO speed about 10MB/s. This case makes the whole speed of swift very slow.
Once I stop the object-auditor process, the speed of swift becomes very quick.

I knew that object-auditor process's function is to ensure the integrity of objects/container/account, but is it neccessary to keep it always on? Perhaps there is something wrong of my configure. So I need some advice and help, Thanks!

Question information

English Edit question
OpenStack Object Storage (swift) Edit question
No assignee Edit question
Solved by:
David Goetz
Last query:
Last reply:
Best David Goetz (david-goetz) said : #1

You can slow down the object auditor by setting the following configuration fields in the object-auditor section of your object-server.conf file:

files_per_second = 20
bytes_per_second = 10000000

The above values are the defaults. You can see this in the docs here:

It is important to have the auditor running to check for data corruption but using the fields above you should be able to set it to a speed that won't affect performance of the cluster too much.

Another another important audit check to run is described here:

The zero byte check can be run at a much faster speed and causes less load because it only looks at zero byte files and doesn't read through the entire files for an md5 for verification of the file's etag. The zero byte file is much more common problem. We run it after we have system failures on a object-server.

mhy (mhy19) said : #2

Thanks David Goetz, that solved my question.

mhy (mhy19) said : #3

Thanks David, I have changed "bytes_per_second = 3000000" and tested the speed of swift. This change indeed can accelerate its performance.