breakdown of the timestamp required for data input (for FAQ)?

Asked by Nathanael Anderson

initial data
2008-07-03 08:41

Target Format
1215088940

what positions should my initial data numbers move to the target format, grabbed the current timestamp within a second of the data in target being submitted, and am having a hard time seeing how they all align, and a full post on how to format the timestamps would clear this up.

Question information

Language:
English Edit question
Status:
Solved
For:
Graphite Edit question
Assignee:
No assignee Edit question
Solved by:
chrismd
Solved:
Last query:
Last reply:
Revision history for this message
chrismd (chrismd) said :
#1

The timestamp is in UNIX epoch time format. This is a single integer counting the number of seconds since January 1st, 1970. It is a standard way of describing time in program on UNIX systems so virtually every programming language provides a way to convert a date/time to this format. If you tell me more about how you are collecting your data (is it a python script?) I would be happy to help in converting your timestamps to epoch form.

Revision history for this message
Nathanael Anderson (wirelessdreamer) said :
#2

i'm writing a perl script to convert data in a textfile in the format of

2008-07-03 08:41 20 datasource

to

datasource_5min 20 CONVERTED_EPOC_TIME

and then open a socket connection to the listener and post all that data.

I'm planning on submitting it, and a standard value collection script in perl if you have any interest in including them with graphite.

a date and time in the format of "2008-07-03 08:41" to the epoch time format. i'm migrating some data from gnu plot scripts, that we have data files from the past few months for, and want to plug them into the graphite.

Revision history for this message
Best chrismd (chrismd) said :
#3

Below is a simple perl script that reads lines formatted like the one you specified and prints them out in the format that graphite requires. Once you finish your script I'd love to see it and possibly include it in future Graphite packages.

#!/usr/bin/perl

use Time::Local;

open(DATA, "my_data_file");

foreach $line (<DATA>) {
  chomp $line;
  my($date, $time, $value, $source) = split(/\s+/, $line);
  my($hour, $minute) = split(/:/,$time);
  my($year, $month, $mday) = split(/-/,$date);
  my $epoch_time = timelocal(0, $minute, $hour, $mday, $month, $year);

  print "$source $value $epoch_time\n";
}

Revision history for this message
Nathanael Anderson (wirelessdreamer) said :
#4

works great, I was able to reformat all the old data, i'm trying this to post the data, is the carbon agent more then just a listening socket, because i'm having trouble posting with netcat.

echo -e "machine.cpsmachine45 20 1213444500\n" | nc localhost 2003

Revision history for this message
Lance Norskog (lance-norskog) said :
#5

My time format is (unfortunately) month/day/year hour:minute:second GMT. This script had an off-by-one error with the month. Here is the version that reads my format:

#!/usr/bin/perl

use Time::Local;

open(DATA, "logs10_date.gr");

foreach $line (<DATA>) {
  chomp $line;
  my($source, $value, $date, $time) = split(/\s+/, $line);
  my($hour, $minute, $second) = split(/:/,$time);
  my($month, $mday, $year) = split(/\//,$date);
  my $epoch_time = timegm($second, $minute, $hour, $mday, $month-1, $year);

  print "$source $value $epoch_time\n";
}

Otherwise, thank you for archiving this five-year-old conversation :)