Graphite

How do I configure carbon to store metrics redundantly?

Asked by Norbert Winkler

I have the following issue:
I want to collect data from several locations (or servers).
Now I want to store all collected data locally at that location (via carbon-cache, storage-schemas and so on),
but in addition to that I want to aggregate (carbon-aggregator) this information (to reduce network load)
and send it to another (main or central) location where all the important metrics from different locations
are stored.
The reason for this is, that internet connection to several locations is not always that reliable as some locations are in developing countries. Therefore a local backup with all the data would reduce the risk of losing important data.
I thought of a setup where I use carbon-relay to duplicate the data. One goes into carbon-cache, the other one into carbon-aggregator which forwards it to the main location.
Is this even possible? How would I achieve this configuration? Or what would be another possibility to achieve my goal.
I appreciate any help.

Question information

Language:
English Edit question
Status:
Solved
For:
Graphite Edit question
Assignee:
No assignee Edit question
Solved by:
Michael Leinartas
Solved:
Last query:
Last reply:
Revision history for this message
Norbert Winkler (norbert-winkler) said : 		#1

help anyone?

What I don't understand is how to configure the destinations-values.
In carbon.conf there are destinations settings under [relay] and under [aggregator].
And in relay-rules.conf you can also set a list of destinations for carbon-relay.
How do I have to set the settings to get the following dataflow (all on one host)?

data-source ---> carbon-relay ---> carbon-aggregator ---> carbon-cache on other host
                                    |
                                    ----------> carbon-cache

Revision history for this message
Best Michael Leinartas (mleinartas) said : 		#2

Let's call it server A and server B

Server A's carbon.conf has:
[carbon-relay]
  RELAY_METHOD = rules
  DESTINATIONS = localhost:2004, localhost:2024 # carbon-cache and carbon-aggregator locally
[carbon-aggregator]
  DESTINATIONS = serverB:2004

Server A's relay-rules.conf will have rules for metrics you want sent to both destinations with:
destinations = localhost:2004, serverB:2004

at the end of the file, you'll put a default rule with:
destinations = localhost:2004

To catch any metrics that weren't sent to the aggregator (and off to serverB)

The basic thing to keep in mind with the relay in relay-rules mode is that *all* destinations you will send to must be configured in carbon.conf under [relay]. relay-rules.conf is where the metrics will get split up into one destination or the other.

Revision history for this message
Norbert Winkler (norbert-winkler) said : 		#3

Thank you very much.
The last paragraph was important for me.
I just tried it out and it works for me.