Graphite

Backing up Whisper files: Best Practices

Asked by Michael Baird

I was looking at the responses to an earlier question asked on the topic of backing up Whisper files on 2010-11-22
https://answers.launchpad.net/graphite/+question/135119

I wanted to see if there are any better practices for accomplishing this now almost 2 years later with the current version of Graphite.

Thanks,
Michael

Question information

Language:
English Edit question
Status:
Answered
For:
Graphite Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Nicholas Leskiw (nleskiw) said : 		#1

I've heard of users switching to SSDs to increase I/O performance, which may negate your need to use a RAM Disk for 150K metrics. Other than that, just copy the files...

Revision history for this message
Francois Mikus (fmikus) said : 		#2

The way I see it you have two ways to do it:

Full copy, nicing the process and disk-access using rsync.

Or :

Have two servers with raids, and simply send your metrics to both servers.
Have a script that periodically(or after an outage) goes through the list of metrics

The script does two things:
1 - On server A checks if the file exists on server B and does an rsync if missing
2 - if not run a whisper-merge to fill in ay missing values from A to B
3 - Do some type of QA and remove the backup file (or edit whisper-merge to not do a backup).

This backs up from A to B.

You can also do the same from B to A, but you have to really trust whisper-merge...

This is simple, and covers most a large number of use cases. Synchronisation is time delayed, but you have redundancy. Graphite sprays requests to both sets of databases.

If anyone else has suggestions, I am also interested in earing the answer.

Revision history for this message
Francois Mikus (fmikus) said : 		#3

.. Hearing the answer ..

Can you help with this problem?

Provide an answer of your own, or ask Michael Baird for more information if necessary.

To post a message you must log in.