I can't make infinite lines drawn with drawAsInfinite() appear if I look at more than 24 hours of logs - should they?

Could be a few things, what is your retention configuration for the affected metrics? (ie. whisper-info.py $wspfile)

This question was expired because it remained in the 'Needs information' state without activity for the last 15 days.

Apologies for the delay - here is the output from whisper-info.py:

maxRetention: 283824000
xFilesFactor: 0.5
fileSize: 210280

Archive 0
retention: 86400
secondsPerPoint: 60
points: 1440
size: 17280
offset: 76

Archive 1
retention: 604800
secondsPerPoint: 300
points: 2016
size: 24192
offset: 17356

Archive 2
retention: 1814400
secondsPerPoint: 900
points: 2016
size: 24192
offset: 41548

Archive 3
retention: 31536000
secondsPerPoint: 3600
points: 8760
size: 105120
offset: 65740

Archive 4
retention: 283824000
secondsPerPoint: 86400
points: 3285
size: 39420
offset: 170860

Your finest precision archive stops at 24h, and your xFilesFactor is 0.5 (which is the default). The xFilesFactor is the percentage of datapoints that must be present in an aggregation window of a high precision archive that is being rolled into a lower precision archive. So since your second highest precision archive has 5 minute precision, it considers the oldest 5 datapoints from the higest precision archive each time it tries to do rollup aggregation. If less than 50% of those 5 datapoints are non-null, it propagates a null value to the lower precision archive.

I imagine you are using drawAsInfinite the way many users do, by sending a non-zero datapoint only when something special happens and not sending a value otherwise. If you were to send 0's when nothing happened this problem would go away because there wouldn't be null values and hence the rollup aggregation would work fine. But since it's probably a pain in the butt to do that (it often is), your next best alternative is to lower the xFilesFactor to 0 so the nulls just get ignored.

This sounds terrible, but my suggestion for right now is to manually hack whisper.py :). Seriously, I only suggest this because the alternative is to a) update all the wsp files on your system plus keep up with new wsp files as they get created, and b) you'd have to script the update process since there isn't currently a tool to do it. It's just changing one byte in a bunch of files, but still. If you hack whisper.py as I explain in a moment, you can just wait until the next release when this problem goes away. I have a new branch that is going to become 0.9.10 in about a month that handles this by letting you apply per-metric configurations in the database via rules that basically work like storage-schemas.conf except generalized to apply arbitrary metadata (so in addition to retentions, you can also set the xfilesfactor).

Here's the whisper hack, look in the __readHeader function around line 200 for:

info = {
  'aggregationMethod' : blah balh blah,
  'maxRetention' : blah blah,
  'xFilesFactor' : xff, # change this to 0
  'archives' : blah,
}

Change the xff to 0 and you're all set. If you update/reinstall whisper of course this will go away, but hopefully the next update is to 0.9.10 when you can then re-fix it properly via the new storage-rules.conf.

What is the proper way to do this now that 0.9.10 is released?

I have created the following,

storage-aggregation.conf:

[lines]
pattern = _line$
xFilesFactor = 0
aggregationMethod = max

And the xFilesFactor seems to work:

$ /usr/local/bin/whisper-info.py storage/whisper/stats/massrel/streamer/reconnect_line.wsp
maxRetention: 56700000
xFilesFactor: 0.0
aggregationMethod: max
fileSize: 1965640

Archive 0
retention: 6048000
secondsPerPoint: 60
points: 100800
size: 1209600
offset: 40

Archive 1
retention: 56700000
secondsPerPoint: 900
points: 63000
size: 756000
offset: 1209640

But when I set that stat I see it via the API (just constantly GETing the JSON for that stat) for a few seconds, and then it zeros out. I can never get a line to stay for more than a few seconds.

Thanks in advance.

Sorry, I'm not sure if I was supposed to select "Add Information Request" before. I hope this notifies the correct people now.