ceph driver for gnocchi crashes

Asked by Giuseppe Attardi on 2017-10-07

I have:
 gnocchi 4.0.3
 cradox-1.3.2
 ceph version 10.2.9

gnocchi-api crashes with this message:

2017-10-07 18:15:16,386 [11903] INFO gnocchi.storage.common.ceph: Ceph storage backend use 'cradox' python library
2017-10-07 18:15:16,709 [11894] INFO gnocchi.cli: Joined coordination group: gnocchi-processing
2017-10-07 18:15:16,716 [11897] INFO gnocchi.cli: Joined coordination group: gnocchi-processing
2017-10-07 18:15:16,718 [11896] INFO gnocchi.cli: Joined coordination group: gnocchi-processing
2017-10-07 18:15:16,722 [11893] INFO gnocchi.cli: Joined coordination group: gnocchi-processing
2017-10-07 18:15:46,386 [11901] ERROR gnocchi.cli: Unexpected error during pending measures reporting
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/gnocchi/cli.py", line 169, in _run_job
    report = self.incoming.measures_report(details=False)
  File "/usr/local/lib/python2.7/dist-packages/gnocchi/storage/incoming/_carbonara.py", line 112, in measures_report
    metrics, measures, full_details = self._build_report(details)
  File "/usr/local/lib/python2.7/dist-packages/gnocchi/storage/incoming/ceph.py", line 117, in _build_report
    i, marker=marker, limit=self.Q_LIMIT))
  File "/usr/local/lib/python2.7/dist-packages/gnocchi/storage/incoming/ceph.py", line 139, in _list_keys_to_process
    op, self.get_sack_name(sack), flag=self.OMAP_READ_FLAGS)
  File "cradox.pyx", line 445, in cradox.requires.wrapper.validate_func (cradox.c:4401)
  File "cradox.pyx", line 3060, in cradox.Ioctx.operate_read_op (cradox.c:38177)
TimedOut: Failed to operate read op for oid incoming128-0
2017-10-07 18:15:46,727 [11893] ERROR gnocchi.cli: Unexpected error processing assigned job
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/gnocchi/cli.py", line 253, in _run_job
    metrics = in_store.list_metric_with_measures_to_process(s)
  File "/usr/local/lib/python2.7/dist-packages/gnocchi/storage/incoming/ceph.py", line 161, in list_metric_with_measures_to_process
    sack, marker=marker, limit=self.Q_LIMIT))
  File "/usr/local/lib/python2.7/dist-packages/gnocchi/storage/incoming/ceph.py", line 139, in _list_keys_to_process
    op, self.get_sack_name(sack), flag=self.OMAP_READ_FLAGS)
  File "cradox.pyx", line 445, in cradox.requires.wrapper.validate_func (cradox.c:4401)
  File "cradox.pyx", line 3060, in cradox.Ioctx.operate_read_op (cradox.c:38177)
TimedOut: Failed to operate read op for oid incoming128-0

Question information

Language:
English Edit question
Status:
Open
For:
Ubuntu Edit question
Assignee:
No assignee Edit question
Last query:
2017-10-07
Last reply:

Can you help with this problem?

Provide an answer of your own, or ask Giuseppe Attardi for more information if necessary.

To post a message you must log in.