Graphs continuously rescaling and shifting

So this is actually handled in Graphite's rendering code (webapp/web/render/glyph.py). When you want to draw more data points than you have pixels then Graphite will consolidate the series by grouping data points and applying a consolidation function to them (the default is to average them).

For example, if you want to draw 1,000 data points on a 500 pixel wide canvas, Graphite will take the average of the 1st and 2nd points to create a consolidated point for the 1st pixel, the 3rd and 4th points average to give the 2nd pixel, etc. You can change the consolidation behavior from averaging to summing by using the cumulative() function in your target, like http://.../render?target=cumulative(foo.bar).

So if you're graph is looking weird it is probably because averaging is not a natural way to combine your data points. If you cannot get your graph to look OK using cumulative(), let me know, we may need to add another consolidation function.

I'm still having problems.

Since the default is to average the numbers, if I update the graph every 2 seconds, I would expect the graph to look better but it's still shifting.

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of chrismd
Sent: Thursday, August 07, 2008 11:14 AM
To: Eric Yeh
Subject: Re: [Question #41517]: Graphs continuously rescaling and shifting

Question #41517 on Graphite changed:
https://answers.launchpad.net/graphite/+question/41517

    Status: Open => Answered

chrismd proposed the following answer:
So this is actually handled in Graphite's rendering code
(webapp/web/render/glyph.py). When you want to draw more data points
than you have pixels then Graphite will consolidate the series by
grouping data points and applying a consolidation function to them (the
default is to average them).

For example, if you want to draw 1,000 data points on a 500 pixel wide
canvas, Graphite will take the average of the 1st and 2nd points to
create a consolidated point for the 1st pixel, the 3rd and 4th points
average to give the 2nd pixel, etc. You can change the consolidation
behavior from averaging to summing by using the cumulative() function in
your target, like http://.../render?target=cumulative(foo.bar).

So if you're graph is looking weird it is probably because averaging is
not a natural way to combine your data points. If you cannot get your
graph to look OK using cumulative(), let me know, we may need to add
another consolidation function.

--
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.

Can you be a little more specific? Is the graph spiky when you are expecting it to be smooth? Could you send me a sample image and its corresponding data? (you can get the raw data by adding a rawData=true query string parameter to the graph url).

Sorry. Attached is the data for two graphs (a.txt and b.txt)

I didn't know how to get the corresponding images for the data at that exact second and was hoping that you could render them from the data.
I am also attaching two images that show what we are seeing (The images are one second apart and you'll notice the y-scaling is different).

The retention settings are 1:432000.

Please let me know if there's anything else I could do.

Thanks,
Eric

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of chrismd
Sent: Thursday, August 07, 2008 2:20 PM
To: Eric Yeh
Subject: RE: [Question #41517]: Graphs continuously rescaling and shifting

Your question #41517 on Graphite changed:
https://answers.launchpad.net/graphite/+question/41517

    Status: Open => Needs information

chrismd requested for more information:
Can you be a little more specific? Is the graph spiky when you are
expecting it to be smooth? Could you send me a sample image and its
corresponding data? (you can get the raw data by adding a rawData=true
query string parameter to the graph url).

--
To answer this request for more information, you can either reply to
this email or enter your reply at the following page:
https://answers.launchpad.net/graphite/+question/41517

You received this question notification because you are a direct
subscriber of the question.

So I've looked into the example data you sent me and I think I see why this is happening.

Basically the Y-axis scaling code looks at the range of Y-values (the max value minus the min value) and tries to divide that range up into a "good" number of "pretty" steps (the step is the difference between the Y-labels) that cover slightly more than the range. What is happening is that one second your range fits well with a certain stepping, and the next second your range changes and the code finds a better stepping. Since the rendering code has no knowledge of your previous graph, the only way to force it to stay within certain parameters is to explicitly specify them.

Currently there is no parameter that tells the rendering engine how to scale the Y-axis, but it would not be hard to add, and that is the only way I see to fix this problem. I will create a bug to track this.

Great. I was wondering if there was a way to fix the y-axis.

The only outstanding issue is the flickering of the graph due to the
number of data points being greater than the number of pixels. As the
data compresses the points, we are getting different averages.

Sent from my iPhone

On Aug 7, 2008, at 6:15 PM, "chrismd" <<email address hidden>
 > wrote:

> Question #41517 on Graphite changed:
> https://answers.launchpad.net/graphite/+question/41517
>
> Status: Open => Answered
>
> chrismd proposed the following answer:
> So I've looked into the example data you sent me and I think I see why
> this is happening.
>
> Basically the Y-axis scaling code looks at the range of Y-values (the
> max value minus the min value) and tries to divide that range up
> into a
> "good" number of "pretty" steps (the step is the difference between
> the
> Y-labels) that cover slightly more than the range. What is happening
> is
> that one second your range fits well with a certain stepping, and the
> next second your range changes and the code finds a better stepping.
> Since the rendering code has no knowledge of your previous graph, the
> only way to force it to stay within certain parameters is to
> explicitly
> specify them.
>
> Currently there is no parameter that tells the rendering engine how to
> scale the Y-axis, but it would not be hard to add, and that is the
> only
> way I see to fix this problem. I will create a bug to track this.
>
> --
> You received this question notification because you are a member of
> graphite-dev, which is an answer contact for Graphite.

So I've committed the changes to control the Y-axis scaling properties, however I'm not sure there is anything to fix for the flickering issue. The reason I say this is because you are storing data with 1-second precision, and expecting to see it with 1-second precision, but are trying to view it in a graph that cannot fit the data. The solution would either be to A) look at a smaller time range, B) increase the graph width, or C) use lower-precision storage.