Graphite

Suppress metrics when count falls below threshold

Asked by Mike

Hello -

I'm tracking some metrics using the percent function.

asPercent( error-count,request-count )

This allows me to determine the error rate.

My issue is that at times my "request-count" metric is very low, meaning a small number of errors causes the overall error rate percentage to spike. I do not want to record a high error rate when the "request-count" metric is below a certain 'threshold'. I'd rather suppress that high error rate. E.g. (if there was an excel like "if" function):

if( request-count < threshold, 0, asPercent( error-count,request-count ))

I toyed around with averageAbove like so:

asPercent( error-count, averageAbove( request-count, threshold ))

Problem with this is you end up with asPercent( error-count, null ) - which asPercent doesn't like. Using transformNull doesn't seem to help either.

Thanks,
Mike

I'm using graphite v. 0.9.10.

Question information

Language:
English Edit question
Status:
Solved
For:
Graphite Edit question
Assignee:
No assignee Edit question
Solved by:
Jason Dixon
Solved:
Last query:
Last reply:
Revision history for this message
Jason Dixon (jason-dixongroup) said : 		#1

Without knowing how you intend to consume the data, you could use the removeAboveValue() function which excludes any values above the defined threshold.

http://graphite.readthedocs.org/en/0.9.12/functions.html#graphite.render.functions.removeAboveValue

Example targets:
asPercent(collectd.graphite.memory.memory-used, sumSeries(collectd.graphite.memory.memory-*))
removeAboveValue(asPercent(collectd.graphite.memory.memory-used, sumSeries(collectd.graphite.memory.memory-*)),14.4)

You can see the resulting graph here:
http://f.cl.ly/items/0S1O1w2L2s1a06293z0p/127.0.0-2.png

Revision history for this message
Mike (mike-lenner) said : 		#2

Hi Jason -

Thanks for the reply.

I think this doesn't accomplish my goal though. Could be I'm misunderstanding.

Using your example, what I'd really like to do is exclude the overall metric if "sumSeries(collectd.graphite.memory.memory-*)" was below a certain threshold.

Put another way, asPercent contains a numerator and denominator effectively.

quotient = asPercent( numerator, denominator )

Your solution excluded the metric if the quotient is above a certain value. What I really want to do is exclude the metric if specifically the denominator is above a certain value.

Revision history for this message
Best Jason Dixon (jason-dixongroup) said : 		#3

Exclude the metric (series) or exclude the datapoint? You can use removeAboveValue() anywhere in the equation you wish, but I suspect you'll get a NaN if you use it in the denominator.

Revision history for this message
Mike (mike-lenner) said : 		#4

Exclude the datapoint only.

Yeah - NaN is the behavior I see. Was searching for a way to give it an alternative value perhaps - e.g. alternativeWhenAboveValue( metric, threshold, alternative ). Perhaps not to be. Thanks for the help.