How to delete a single Whisper entry?

Asked by Michael Alford on 2012-05-17

How can I delete a single Whisper record so that the particular metric for a particular time interval is as-if it was never entered, like it would be if the Whisper database had just been created.

While I cannot find a "delete record" function in Graphite or Carbon, one thought was to use Carbon's text socket interface to set the value to a null value.

With Graphite v0.9.9, I've tried using the Carbon text interface to set a record's value to "null", "None", and "NaN", but the record's numeric value persists.

Question information

Language:
English Edit question
Status:
Solved
For:
Graphite Edit question
Assignee:
No assignee Edit question
Solved by:
Michael Alford
Solved:
2012-05-25
Last query:
2012-05-25
Last reply:
2012-05-17
Michael Leinartas (mleinartas) said : #1

Unfortunately there's no way to do this right now. The Whisper format stores points as pairs of (<timestamp>,<value>) at predictable offsets. None is returned when the timestamp stored doesn't match the timestamp expected at that offset, i.e. that point was never written to.

The only way currently to reset these values would be to recreate the whisper file and re-store the existing values, filtering out the ones you dont want. This can be done (though painfully) with whisper-fetch.py and whisper-update.py, or by scripting against the whisper Python module.

See here for a bit more detail on the file format: readthedocs.org/docs/graphite/en/latest/whisper.html

Given that there is no built-in solution to this, I am attempting to modify Carbon and Whisper to accept a value of "None" for a point record.

I added the following function to both whisper/whisper.py and carbon/lib/carbon/protocols.py:

def floatOrNone(value):
  try:
    return float(value)
  except ValueError:
    return None
  except TypeError:
    return None

And then called the above function instead of float() for MetricLineReceiver, MetricDatagramReceiver, MetricPickleReceiver in Carbon and in the update and update_many functions of Whisper.

Whisper would then fail when trying to pack a None "value" in pointFormat, so I changed a portion of the __archive_update_many function to:
    if (not previousInterval) or (interval == previousInterval + step):
      if (value == None):
        currentString += struct.pack(pointFormat,0,0)
      else:
        currentString += struct.pack(pointFormat,interval,value)
      previousInterval = interval
    else:
      numberOfPoints = len(currentString) / pointSize
      startInterval = previousInterval - (step * (numberOfPoints-1))
      packedStrings.append( (startInterval,currentString) )
      if (value == None):
        currentString = struct.pack(pointFormat,0,0)
      else:
        currentString = struct.pack(pointFormat,interval,value)

I am now getting no error but I am still failing to delete the record. It seems that saving a zero timestamp in a place in the database that should have a different timestamp should create a situation where the point record is effectively deleted, based on Michael Leinartas' comment above about what happens when a timestamp mismatch occurs.

Changing status of this question to solved, as the question was correctly answered as "not supported".

Someday, I would like to investigate further into adding this functionality into Carbon and Whisper, but for now I am tabling such efforts.