How can I increase the storage time in carbon?

Asked by edanshalom

I'm trying to insert data to carbon using metric_path, value and a timestamp.
However, I can insert data only 7 days back.
Is it because of the configurations in storage-schemas.conf?
I noticed it was:
[default]
pattern = .*
retentions = 10s:7d

I changed it to:
[default]
pattern = .*
retentions = 10s:30d

But it didn't help.
So how can I increase the storage time in carbon?
Thanks.

Question information

Language:
English Edit question
Status:
Solved
For:
Graphite Edit question
Assignee:
No assignee Edit question
Solved by:
edanshalom
Solved:
Last query:
Last reply:
Revision history for this message
Denis Zhdanov (denis-zhdanov) said :
#1

You need to recreate your whisper files after changing retention scheme. Or
you can use whisper-resize utility probably.
On Mon, 16 Nov 2015 at 22:47, edanshalom <
<email address hidden>> wrote:

> New question #274448 on Graphite:
> https://answers.launchpad.net/graphite/+question/274448
>
> I'm trying to insert data to carbon using metric_path, value and a
> timestamp.
> However, I can insert data only 7 days back.
> Is it because of the configurations in storage-schemas.conf?
> I noticed it was:
> [default]
> pattern = .*
> retentions = 10s:7d
>
> I changed it to:
> [default]
> pattern = .*
> retentions = 10s:30d
>
> But it didn't help.
> So how can I increase the storage time in carbon?
> Thanks.
>
>
> --
> You received this question notification because your team graphite-dev
> is an answer contact for Graphite.
>
> _______________________________________________
> Mailing list: https://launchpad.net/~graphite-dev
> Post to : <email address hidden>
> Unsubscribe : https://launchpad.net/~graphite-dev
> More help : https://help.launchpad.net/ListHelp
>

Revision history for this message
edanshalom (edanshalom) said :
#2

I recreate my whisper files each time I run graphite.
I create a docker image based on the following repo:
https://github.com/kamon-io/docker-grafana-graphite

The following scheme would give me 30d storage with a 10s precision?
[default]
pattern = .*
retentions = 10s:30d

I don't have any whisper files when I create the image.
I build it after each change from scratch.

It's important to say again that I'm trying to insert the data to carbon using metric_path, value and a timestamp.
Not through statsd, directly to carbon. Maybe this is the problem.
I insert the data in the following way:
http://coreygoldberg.blogspot.co.il/2012/04/python-getting-data-into-graphite-code.html

Revision history for this message
Denis Zhdanov (deniszhdanov) said :
#3

If you recreating your whisper every time - then it should be enough.
Yes, your new scheme would give you 30d storage with a 10s precision, exactly. And if you following manual for inserting data above - it should work.
Could you little bit explain more what's going on? Why do you think that you can insert data only 7 days back ?

Revision history for this message
edanshalom (edanshalom) said :
#4

It's working now.
I recreated it again and it just worked.
Don't know what may have caused the problems.

Is it possible to define for different periods different precisions?
for example:
7d storage with a 10s precision
30d storage with a 1m precision
1y storage with a 1d precision

What will be the syntax?
Thanks.

Revision history for this message
Denis Zhdanov (deniszhdanov) said :
#5

>Is it possible to define for different periods different precisions?
Of course, just add more comma delimited precisions, like

...
retentions = 10s:7d, 1m:30d, 1d:1y

Just be sure that precision of a longer retention archive is cleanly divisible by precision of next lower retention archive. E.g.
in your case, 1m / 10s = 6 - OK, 1d / 1m = 1440 - OK.

Revision history for this message
edanshalom (edanshalom) said :
#6

Awesome! Thanks!