Unable to post tags

Asked by Tobias on 2018-01-15

Hello!

I have installed graphite 1.1.1 and I am trying to post tags to the graphite database according to the documentation:

http://graphite.readthedocs.io/en/latest/tags.html#adding-series-to-the-tagdb

Here is the output of what I have done:

1.
# grep graphite /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 graphite

2.
# echo "disk.used 300 $(date +%s)" | nc graphite 2003

3.
# curl "http://graphite/render?&target=disk.used&from=-5min&rawData=true"
disk.used,1516009980,1516010280,60|300.0,None,None,300.0,300.0

4.
# curl -i -X POST "http://graphite/tags/tagSeries" --data-urlencode 'path=disk.used;rack=a1;datacenter=dc1;server=web01'

HTTP/1.1 500 Internal Server Error
Date: Mon, 15 Jan 2018 09:57:36 GMT
Server: Apache/2.4.25 (Red Hat) mod_wsgi/4.5.13 Python/2.7
Content-Length: 37
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, OPTIONS
Access-Control-Allow-Headers: origin, authorization, accept
Connection: close
Content-Type: application/json

{"error": "near \",\": syntax error"}

So graphite exists in /etc/hosts (point 1)
I can post metrics to graphite and retrieve them (point 2 & 3)
But it seems graphite is not accepting the tags I am trying to post. (point 4)

Do you know what I am missing here?

Kind Regards,

Toby

Question information

Language:
English Edit question
Status:
Solved
For:
Graphite Edit question
Assignee:
No assignee Edit question
Solved by:
Tobias
Solved:
2018-01-19
Last query:
2018-01-19
Last reply:
2018-01-15
Denis Zhdanov (deniszhdanov) said : #1

As documentation says "Normally carbon will take care of this, it submits all-new series to the TagDB, and periodically re-submits all series to ensure that the TagDB is kept up to date". Did you tried to submit metric in carbon?
Like
echo "disk.used;rack=a1;datacenter=dc1;server=web01 300 $(date +%s)" | nc localhost 2003
?

Also `{"error": "near \",\": syntax error"}` is very strange error, there's no ',' symbol in your request. I tried to do the same on 1.1.1 from docker image - it works fine.

Tobias (lindqt01) said : #2

# echo "disk.used;rack=a1;datacenter=dc1;server=web01 600 $(date +%s)" | nc graphite 2003

# curl -s "http://graphite/tags?pretty=1"
[
  {
    "id": 1,
    "tag": "name"
  }
]

Does not seem to register the tags. And the metrics does not show either:

# curl "http://graphite/render?&target=disk.used&from=-5min&rawData=true"
disk.used,1516016460,1516016760,60|600.0,None,None,None,None

Yeah it seems strange.

Denis Zhdanov (deniszhdanov) said : #3

What's GRAPHITE_URL in your carbon.conf ?
Should pointed to Graphite, e.g.
GRAPHITE_URL = http://graphite:80

Tobias (lindqt01) said : #4

GRAPHITE_URL = http://127.0.0.1

Is that OK?

Tobias (lindqt01) said : #6

[root@localhost ~]# echo "disk.used;rack=a1;datacenter=dc1;server=web01 600 $(date +%s)" | nc graphite 2003
[root@localhost ~]# echo "disk.used;rack=a1;datacenter=dc1;server=web01 600 $(date +%s) bogus" | nc graphite 2003

Corresponding lines in carbon listener.log file:

16/01/2018 10:33:43 :: MetricLineReceiver connection with 127.0.0.1:53046 established
16/01/2018 10:33:43 :: MetricLineReceiver connection with 127.0.0.1:53046 closed cleanly
16/01/2018 10:33:48 :: MetricLineReceiver connection with 127.0.0.1:53048 established
16/01/2018 10:33:48 :: invalid line received from client 127.0.0.1:53048, ignoring [disk.used;rack=a1;datacenter=dc1;server=web01 600 1516095228 bogus]
16/01/2018 10:33:48 :: MetricLineReceiver connection with 127.0.0.1:53048 closed cleanly

So it seems to work but no metrics and no tags are showing.

Tobias (lindqt01) said : #7

Yes it seems to be working:

# curl -v "http://127.0.0.1/render?&target=disk.used&from=-5min&rawData=true"
* About to connect() to 127.0.0.1 port 80 (#0)
* Trying 127.0.0.1... connected
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
> GET /render?&target=disk.used&from=-5min&rawData=true HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.21 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: 127.0.0.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Tue, 16 Jan 2018 09:36:11 GMT
< Server: Apache/2.4.25 (Red Hat) mod_wsgi/4.5.13 Python/2.7
< Content-Length: 60
< Expires: Tue, 16 Jan 2018 09:37:11 GMT
< Cache-Control: max-age=60
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Methods: GET, OPTIONS
< Access-Control-Allow-Headers: origin, authorization, accept
< Content-Type: text/plain; charset=UTF-8
<
disk.used,1516095120,1516095420,60|None,None,None,None,None
* Connection #0 to host 127.0.0.1 left intact
* Closing connection #0

Denis Zhdanov (deniszhdanov) said : #8

Do you have `/opt/graphite/storage/whisper/_tagged` directory created?
Does carbon was also upgraded to 1.1.x version?

Everything is looks fine, should be working.

Tobias (lindqt01) said : #9

Hi, yes that directory exists. Here is what I see in it:

/opt/vgt/graphite/storage/whisper/_tagged/e9a/90f/disk_DOT_used\;datacenter\=dc1\;rack\=a1\;server\=web01.wsp

I can also see the _tagged directory in the graphite-web browser.

Maybe it is because I am installing graphite in a custom location:

/opt/vgt/graphite

Denis Zhdanov (deniszhdanov) said : #10

Tags data is stored in database, path shouldn't have matter

Tobias (lindqt01) said : #11

The problem was that I was running carbon-c-relay in front. My appologies for wasting your time. Thanks very much for your support.

Denis Zhdanov (deniszhdanov) said : #12

It should be compatible, but you'll probably need to fix validation, indeed

Piotr Popieluch (piotr1212) said : #13

There is work done on tag support in c-relay. It needs testing:

https://github.com/grobian/carbon-c-relay/issues/311