Graphite

unexpected spikes being reported

Asked by Gary Levin

I have an application reporting metrics to statsd every 5 seconds. I expect to see the values plotted in Graphite. They are showing up, but occasionally there is a spike that is not reflected in the data that I log when sending to statsd.

Every 5 seconds I send a number in the range 50-300, reported as a gauge. The data coming out of graphite is usually in that range, but at random intervals it reports a value around 10M. My retention is 30s:1h,1m:2h,10m:1d, with statsd flushing every 10 seconds. The problem is visible in the last 10 minutes, so it is from the finest granularity stored.

I am just starting to use Graphite and expect I have something misconfigured. I am still using the default aggregation settings.

Any suggestion would be appreciated. Obvious configuration problems or pointing out logs to check to see that the data is being received correctly and how it is being aggregated.

Question information

Language:
English Edit question
Status:
Expired
For:
Graphite Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Gary Levin (gmlevin) said : 		#1

I think I see the problem, but I will still need some help fixing it.

The statsd client (python statsd) is producing udp packets like "X:100|gY:200|g" and statsd+graphite is interpreting it as X=200 and ignoring Y.

So options:
* the statsd client is creating ill-formed udp packets.
* the statsd daemon is not interpreting them correctly

If there were a way to see what is being sent to / received by graphite, I could close this issue and move the problem to one of the statsd packages.

Revision history for this message
Piotr Popieluch (piotr1212) said : 		#2

Easiest way is to "tcpdump" or configure statsd to a port which runs "nc -k -l".

Revision history for this message
Launchpad Janitor (janitor) said : 		#3

This question was expired because it remained in the 'Open' state without activity for the last 15 days.