OpenStack Object Storage (swift)

object-auditor process consume disk I/O

Asked by mhy

hello! my swift is 1 proxy node + 5 zones(one storage node each zone, each storage node has 2T disk space), siwft1.4.3

on my storage node, the disk IO speed I got with the help of "time dd if=/dev/zero of=/srv/node/sda3/test bs=1M count=2000" is about 120MB/s;
But, once the object-auditor process of storage nodes is on, the IO speed will decrease violently, because the object-auditor has been always reading disk and generating read IO speed about 10MB/s. This case makes the whole speed of swift very slow.
Once I stop the object-auditor process, the speed of swift becomes very quick.

I knew that object-auditor process's function is to ensure the integrity of objects/container/account, but is it neccessary to keep it always on? Perhaps there is something wrong of my configure. So I need some advice and help, Thanks!

Question information

Language:
English Edit question
Status:
Solved
For:
OpenStack Object Storage (swift) Edit question
Assignee:
No assignee Edit question
Solved by:
David Goetz
Solved:
Last query:
Last reply:
Revision history for this message
Best David Goetz (david-goetz) said : 		#1

You can slow down the object auditor by setting the following configuration fields in the object-auditor section of your object-server.conf file:

files_per_second = 20
bytes_per_second = 10000000

The above values are the defaults. You can see this in the docs here:
http://swift.openstack.org/deployment_guide.html#object-server-configuration

It is important to have the auditor running to check for data corruption but using the fields above you should be able to set it to a speed that won't affect performance of the cluster too much.

Another another important audit check to run is described here:
http://swift.openstack.org/admin_guide.html#object-auditor

The zero byte check can be run at a much faster speed and causes less load because it only looks at zero byte files and doesn't read through the entire files for an md5 for verification of the file's etag. The zero byte file is much more common problem. We run it after we have system failures on a object-server.

Revision history for this message
mhy (mhy19) said : 		#2

Thanks David Goetz, that solved my question.

Revision history for this message
mhy (mhy19) said : 		#3

Thanks David, I have changed "bytes_per_second = 3000000" and tested the speed of swift. This change indeed can accelerate its performance.