Graphite SNMP Counter transforms

Hi

I have this exact issue myself, I think I've solved it by using the following,

The Magical 0.125 I'm not sure were this comes from I've seen this around a few places and had simular results as yourself,

the scale has to be calculated on the sample time your doing

Sample (divide by) timeinseconds x 8 (bits) = yourscale

For example,

I'm doing 3 second samples so I worked out my scale like this

1 / 3 = 0.3333333333333333 * 8 = 2.666666666666667

Here is snapshot of one part of my url

alias(scale(nonNegativeDerivative(s3_sw1.snmp.if_octets-GigabitEthernet_0_1.rx),2.666666666666667),'RX')

Also keep in mind you need to setup your whisper storage-schemas.conf as well to expect your data every x seconds this is why I used the prefix "s3_" eg,

[real_time_3s_120days]
pattern = s3.*
retentions = 3s:120d

Hope this helps :)

Regards,
Quenten

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of David Michael Pennington
Sent: Monday, 10 September 2012 9:01 PM
To: <email address hidden>
Subject: [Graphite-dev] [Question #208167]: Graphite SNMP Counter transforms

New question #208167 on Graphite:
https://answers.launchpad.net/graphite/+question/208167

I'm using graphite version 0.9.10 to read ifHCInOctets and ifHCOutOctets, which I'm polling with the collectd snmp and graphite_write plugins; I'm using collectd version 5.1.0.

Data arrives into graphite with no problems; however, I want to graph as bits-per-second. To test whether my stats are correct, I started a CD iso download and watched the download rate... it varied between 1.0Mbps and 2.0Mbps.

Common-sense tells you that you need to multiply Octets counters by 8 to get bits; however, it seems that I need to divide by 8 to make graphite display correctly.

When I multiply by a 0.125 scale factor, "scale(scaleToSeconds(nonNegativeDerivative(<SERIES>), 60),0.125), the formula correctly converts to bits-per-second and I see numbers between 1Mpbs and 2Mbps...

When I use a scale-factor of 8, it fails spectacularly; I get readings of 120Mbps, but I know that's impossible because my circuit is capped at 5Mpbs.

I posted the gory-details (including config files and graph snapshots) here:
http://serverfault.com/questions/425828/graphite-snmp-counter-transforms

Could someone me understand the math such that scale(<series>, 8) fails?

--
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.

_______________________________________________
Mailing list: https://launchpad.net/~graphite-dev
Post to : <email address hidden>
Unsubscribe : https://launchpad.net/~graphite-dev
More help : https://help.launchpad.net/ListHelp

I think that sending the correct value to Graphite in the first place is a much better solution. ;)

Quenten Grasso <email address hidden> wrote:

>Question #208167 on Graphite changed:
>https://answers.launchpad.net/graphite/+question/208167
>
> Status: Open => Answered
>
>Quenten Grasso proposed the following answer:
>Hi
>
>I have this exact issue myself, I think I've solved it by using the
>following,
>
>The Magical 0.125 I'm not sure were this comes from I've seen this
>around a few places and had simular results as yourself,
>
>the scale has to be calculated on the sample time your doing
>
>Sample (divide by) timeinseconds x 8 (bits) = yourscale
>
>For example,
>
>I'm doing 3 second samples so I worked out my scale like this
>
>1 / 3 = 0.3333333333333333 * 8 = 2.666666666666667
>
>Here is snapshot of one part of my url
>
>alias(scale(nonNegativeDerivative(s3_sw1.snmp.if_octets-
>GigabitEthernet_0_1.rx),2.666666666666667),'RX')
>
>Also keep in mind you need to setup your whisper storage-schemas.conf as
>well to expect your data every x seconds this is why I used the prefix
>"s3_" eg,
>
>[real_time_3s_120days]
>pattern = s3.*
>retentions = 3s:120d
>
>Hope this helps :)
>
>Regards,
>Quenten
>
>-----Original Message-----
>From: <email address hidden> [mailto:<email address hidden>] On Behalf Of David Michael Pennington
>Sent: Monday, 10 September 2012 9:01 PM
>To: <email address hidden>
>Subject: [Graphite-dev] [Question #208167]: Graphite SNMP Counter transforms
>
>New question #208167 on Graphite:
>https://answers.launchpad.net/graphite/+question/208167
>
>I'm using graphite version 0.9.10 to read ifHCInOctets and
>ifHCOutOctets, which I'm polling with the collectd snmp and
>graphite_write plugins; I'm using collectd version 5.1.0.
>
>Data arrives into graphite with no problems; however, I want to graph as
>bits-per-second. To test whether my stats are correct, I started a CD
>iso download and watched the download rate... it varied between 1.0Mbps
>and 2.0Mbps.
>
>Common-sense tells you that you need to multiply Octets counters by 8 to
>get bits; however, it seems that I need to divide by 8 to make graphite
>display correctly.
>
>When I multiply by a 0.125 scale factor,
>"scale(scaleToSeconds(nonNegativeDerivative(<SERIES>), 60),0.125), the
>formula correctly converts to bits-per-second and I see numbers between
>1Mpbs and 2Mbps...
>
>When I use a scale-factor of 8, it fails spectacularly; I get readings
>of 120Mbps, but I know that's impossible because my circuit is capped at
>5Mpbs.
>
>I posted the gory-details (including config files and graph snapshots) here:
>http://serverfault.com/questions/425828/graphite-snmp-counter-transforms
>
>Could someone me understand the math such that scale(<series>, 8) fails?
>
>--
>You received this question notification because you are a member of
>graphite-dev, which is an answer contact for Graphite.
>
>_______________________________________________
>Mailing list: https://launchpad.net/~graphite-dev
>Post to : <email address hidden>
>Unsubscribe : https://launchpad.net/~graphite-dev
>More help : https://help.launchpad.net/ListHelp
>
>You received this question notification because you are a member of
>graphite-dev, which is an answer contact for Graphite.
>
>_______________________________________________
>Mailing list: https://launchpad.net/~graphite-dev
>Post to : <email address hidden>
>Unsubscribe : https://launchpad.net/~graphite-dev
>More help : https://help.launchpad.net/ListHelp

Quenten,

FYI, by including a time-conversion factor in your scale() function, you have likely broken data aggregation; see...
http://obfuscurity.com/2012/05/A-Precautionary-Tale-for-Graphite-Users

Hi David

As I understand this is only an issue if your keeping multiple retentions for example,

retentions = 1m:1d,5m:28d,15m:1y

if you were just keeping the single retention it would be fine.

retentions = 1m:1d

However, if someone is happy to share a better solution I'm all ears!

Regards,
Quenten
-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of David Michael Pennington
Sent: Tuesday, 11 September 2012 6:26 PM
To: <email address hidden>
Subject: Re: [Graphite-dev] [Question #208167]: Graphite SNMP Counter transforms

Question #208167 on Graphite changed:
https://answers.launchpad.net/graphite/+question/208167

David Michael Pennington posted a new comment:
Quenten,

FYI, by including a time-conversion factor in your scale() function, you have likely broken data aggregation; see...
http://obfuscurity.com/2012/05/A-Precautionary-Tale-for-Graphite-Users

--
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.

_______________________________________________
Mailing list: https://launchpad.net/~graphite-dev
Post to : <email address hidden>
Unsubscribe : https://launchpad.net/~graphite-dev
More help : https://help.launchpad.net/ListHelp