Graphite

Megacarbon Aggregator Missing AGGREGATION_WRITE_BACK_FREQUENCY Configuration Setting

Asked by Kieran Girvan on 2016-01-22

I've installed the latest megacarbon branch along with my standard Graphite install. I'm trying to get the aggregator up and running but struggling to find much in the way of documentation. I have 2 write carbon daemons and a single relay on running on the same node.

Both of my write carbon daemons are setup as writers only i.e.

#/opt/graphite/conf/carbon-daemons/writer-(1,2)/daemon.conf
PIPELINE = write

My relay is setup to filter, relay and aggregate:

#/opt/graphite/conf/carbon-daemons/relay/daemon.conf
PIPELINE = filter,aggregate,relay

Within my relay aggregation rules I have the following entry:

test.aggregate.all.requests (60) = sum test.aggregate.requests

When I try and submit a metric matching this input mask I receive the following error in the relay console.log:

18/01/2016 11:12:05 :: Unhandled Error
Traceback (most recent call last):
  File "/opt/graphite/lib/carbon/protocols.py", line 74, in lineReceived
    self.metricReceived(metric, datapoint)
  File "/opt/graphite/lib/carbon/protocols.py", line 60, in metricReceived
    events.metricReceived(metric, datapoint)
  File "/opt/graphite/lib/carbon/events.py", line 20, in __call__
    handler(*args, **kwargs)
  File "/opt/graphite/lib/carbon/pipeline.py", line 27, in run_pipeline
    run_pipeline(out_metric, out_datapoint, processors[1:])
--- <exception caught here> ---
  File "/opt/graphite/lib/carbon/pipeline.py", line 25, in run_pipeline
    for out_metric, out_datapoint in processor.process(metric, datapoint):
  File "/opt/graphite/lib/carbon/aggregator/processor.py", line 58, in process
    buffer.configure_aggregation(rule.frequency, rule.aggregation_func)
  File "/opt/graphite/lib/carbon/aggregator/buffers.py", line 57, in configure_aggregation
    self.compute_task.start(settings['AGGREGATION_WRITE_BACK_FREQUENCY'] or int(frequency), now=False)
  File "/opt/graphite/lib/carbon/conf.py", line 116, in __getitem__
    raise ConfigError("Missing expected configuration \"%s\"" % key)
carbon.exceptions.ConfigError: <ConfigError(Missing expected configuration "AGGREGATION_WRITE_BACK_FREQUENCY")>

I've struggling to understand where this configuration setting needs to be placed?

Question information

Language:: English Edit question

Status:: Expired

For:: Graphite Edit question

Assignee:: No assignee Edit question

Last query:: 2016-01-22

Last reply:: 2016-02-07

Link existing bug

Revision history for this message

Kieran Girvan (kierangirvan) said on 2016-01-22:

OK I've made some progress on this, looks like this configuration setting is expected to be set here:

#/opt/graphite/conf/carbon-daemons/relay/aggregation.conf
AGGREGATION_WRITE_BACK_FREQUENCY = 10

Having set this configuration option and restarted the relay daemon I received the following error in the console.log:

22/01/2016 06:26:47 :: Unhandled Error
Traceback (most recent call last):
  File "/opt/graphite/lib/carbon/protocols.py", line 74, in lineReceived
    self.metricReceived(metric, datapoint)
  File "/opt/graphite/lib/carbon/protocols.py", line 60, in metricReceived
    events.metricReceived(metric, datapoint)
  File "/opt/graphite/lib/carbon/events.py", line 20, in __call__
    handler(*args, **kwargs)
  File "/opt/graphite/lib/carbon/pipeline.py", line 27, in run_pipeline
    run_pipeline(out_metric, out_datapoint, processors[1:])
--- <exception caught here> ---
  File "/opt/graphite/lib/carbon/pipeline.py", line 25, in run_pipeline
    for out_metric, out_datapoint in processor.process(metric, datapoint):
  File "/opt/graphite/lib/carbon/aggregator/processor.py", line 58, in process
    buffer.configure_aggregation(rule.frequency, rule.aggregation_func)
  File "/opt/graphite/lib/carbon/aggregator/buffers.py", line 57, in configure_aggregation
    self.compute_task.start(settings['AGGREGATION_WRITE_BACK_FREQUENCY'] or int(frequency), now=False)
  File "/usr/lib64/python2.7/site-packages/twisted/internet/task.py", line 196, in start
    self._scheduleFrom(self.starttime)
  File "/usr/lib64/python2.7/site-packages/twisted/internet/task.py", line 279, in _scheduleFrom
    self.call = self.clock.callLater(howLong(), self)
  File "/usr/lib64/python2.7/site-packages/twisted/internet/task.py", line 265, in howLong
    untilNextInterval = 10 - (runningFor % self.interval)
exceptions.TypeError: unsupported operand type(s) for %: 'float' and 'str'

Looks to be an issue with the configuration value not being parsed as an integer within the buffers.py

Line 57 of this python script had the following declaration parsing in the AGGREGATION_WRITE_BACK_FREQUENCY configuration setting:

self.compute_task.start(settings['AGGREGATION_WRITE_BACK_FREQUENCY'] or int(frequency), now=False)]]

I made a small change to this python script to cast the configuration setting as an integer.

self.compute_task.start(int(settings['AGGREGATION_WRITE_BACK_FREQUENCY']) or int(frequency), now=False)

The aggregation now works.

I'm not sure if this is the best way to fix this possible bug or if I'm setting the configuration incorrectly in the first place.