Generating graph even without all target available

Asked by Alexandre

Hi,

 I have a script that parse http logs, summarize/aggregate the data and send to carbon.

 That scripts send information like number of requests per status code (200, 301, 302, 404, 503), per method (POST, GET, HEAD, etc) and I created graphics that show percentage of each kind of request and error code to follow the numbers.

 Everything works great until I face with a situation where in a certain period of time there isn't any request with a specific status code so there is no data in the whisper database, when this happens my existent graphs are not displayed anymore and the graphite-web display the error "reduce() of empty sequence with no initial value".

 Is there any way to make graphite interpret "no data" as 0 ? Or do I need to change my script to send zero value when there isn´t data in that period ?

Regards,
Alexandre

Question information

Language:
English Edit question
Status:
Answered
For:
Graphite Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Jeff Blaine (jblaine-kickflop) said :
#1

> I have a script that parse http logs, summarize/aggregate the data and send to carbon.
>
> That scripts send information like number of requests per status code (200, 301, 302, 404, 503), per method (POST, GET, HEAD, etc) and I created graphics that show percentage of each kind of request and error code to follow the numbers.
>
> Everything works great until I face with a situation where in a certain period of time there isn't any request with a specific status code so there is no data in the whisper database, when this happens my existent graphs are not displayed anymore and the graphite-web display the error "reduce() of empty sequence with no initial value".
>
> Is there any way to make graphite interpret "no data" as 0 ? Or do I need to change my script to send zero value when there isn´t data in that period ?

http://graphite.readthedocs.org/en/0.9.10/render_api.html#drawnullaszero

?

Revision history for this message
Alexandre (1le) said :
#2

Hi Jeff,

 I´ve looked at drawnullaszero, but it does not solve my problem because I make some calculations with the data... something like:

asPercent(sum(app.prod.*.nginx.domains.www_foo_org.status.204)%2Csum(app.prod.*.nginx.domains.www_foo_org.status.*))

and the function drawnullaszero actually does not "change" the data...

Any other suggestion ?

Revision history for this message
Michael Leinartas (mleinartas) said :
#3

You can try this one instead: http://graphite.readthedocs.org/en/0.9.10/functions.html#graphite.render.functions.transformNull

Note though that it will only work if the series you're wrapping actually has 'None' data - that is, if you request outside of a series's bounds (too far in the past) or request a series that does not exist on disk (or a wildcard that matches nothing) it will not work.

Good luck

Revision history for this message
Jeff Blaine (jblaine-kickflop) said :
#4

Maybe transformNull?

http://graphite.readthedocs.org/en/0.9.10/functions.html#graphite.render.functions.transformNull

I am guessing that you have the the storage schema configured for these http requests to only have 1 retention period? I think it is pretty rare to have carbon return no data values for a whisper database that *does* exist, but that may just be because most people configure more than one retention specification (each of which makes a whisper 'archive').

This may help re: above paragraph?

http://graphite.readthedocs.org/en/0.9.10/whisper.html#multi-archive-storage-and-retrieval-behavior

Revision history for this message
Michael Leinartas (mleinartas) said :
#5

My guess is that there *is* data being returned but it's all None values - many of the functions filter out all values that aren't None and if *all* the values are None the series ends up having no points after this pre-processing. Ultimately this is a bug in the functions themselves which should do their best to never blow up with an exception unless called incorrectly or in an invalid way (i.e. something _exception_al)

But yeah, transformNull should work in that case

Can you help with this problem?

Provide an answer of your own, or ask Alexandre for more information if necessary.

To post a message you must log in.