OpenStack Object Storage (swift)

Howto Multiserver Stats

Asked by Travis W Kiel

Proxy and auth are on their own machine, so obviously SAIO stats example wont work out.

Is the best option for this to ship logs from the proxy over to one of the storage (account/object/container) servers and then run the stats jobs there?

Question information

Language:
English Edit question
Status:
Solved
For:
OpenStack Object Storage (swift) Edit question
Assignee:
No assignee Edit question
Solved by:
Travis W Kiel
Solved:
Last query:
Last reply:
Revision history for this message
John Dickinson (notmyname) said : 		#1

The stats system puts files in to the swift cluster itself and the processing piece pulls from the swift cluster. Therefore all logs are available to every machine in the cluster (every machine that has network access to the storage servers).

The 3 phases of the stats system are:
1) generate logs (done by the proxy servers and swift-account-stats-logger)
2) upload logs to swift (done by swift-log-uploader)
3) process logs (done by swift-log-processor

This means you can have designated log processing machines or use any of your existing machines. Designated machines, of course, will have less impact on standard operations. The stats system is CPU and memory hungry.

Revision history for this message
Travis W Kiel (twkiel) said : 		#2

I re-read the SAIO stats page and I realized I was confusing it. After that, https://bugs.launchpad.net/swift/+bug/671236 finished off the setup.