Graphite

how to handle high-frequency data?

Asked by Gábor Farkas

i'd like to put my web-server access-log data into graphite.
i simply want to put in the duration how long it took the webserver to process the request. the problem is that i have a lot of requests/second, let's say 50requests/second. how should i get this data into graphite? can i send in 50values for every second?
or, should i "summarize" the values for every second (let's say min/max/median/mean), and submit only those values to graphite for every second? what is the recommended approach?

thanks,
gabor

Question information

Language:
English Edit question
Status:
Answered
For:
Graphite Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Nicholas Leskiw (nleskiw) said : 		#1

I would summarize the values, then send them to graphite. That way you can intelligently handle edge cases and you know exactly what your numbers represent.

Revision history for this message
mathieu longtin (mathieu-1) said : 		#2

Read this, this is exactly what you're looking for:

http://codeascraft.etsy.com/2011/02/15/measure-anything-measure-everything/

It would be nice if it was integrated in the carbon server...

Can you help with this problem?

Provide an answer of your own, or ask Gábor Farkas for more information if necessary.

To post a message you must log in.