Graphite

carbon-cache number of cached points is very cyclical

Asked by Nelson Elhage

A graph of carbon.agents.*.cache.size shows that the number of cached points is *very* cyclical, with a period of around 2 hours:
https://nelhage.com/files/stripe-carbon-cache.png

Cross-referencing with munin graphs shows that disk write activity does seem to spike for brief intervals on the same period, suggesting that carbon cache is basically just accumulating data points and then writing them out in bursts. Is this normal? It seems like we can't be nearly utilizing our disk fully with that kind of behavior. Is there something I can tune to make it smooth the disk usage?

Question information

Language:
English Edit question
Status:
Answered
For:
Graphite Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Nicholas Leskiw (nleskiw) said : 		#1

Graphite tries to write data in bursts to the disk. This is due to the nature of spinning disks. Writing sequential data to disk is not much more expensive than writing a single datapoint; most of the time is wasted in the seek. This is why graphite pre-allocates data to disk, to make it as sequential as possible. Then data queues up and many data points are written at once.

Read this question about tuning your carbon cache to use your disk most effectively:

https://answers.launchpad.net/graphite/+question/212303

Can you help with this problem?

Provide an answer of your own, or ask Nelson Elhage for more information if necessary.

To post a message you must log in.