ceph driver for gnocchi crashes

Asked by Giuseppe Attardi on 2017-10-07

I have:
 gnocchi 4.0.3
 cradox-1.3.2
 ceph version 10.2.9

gnocchi-api crashes with this message:

2017-10-07 18:15:16,386 [11903] INFO gnocchi.storage.common.ceph: Ceph storage backend use 'cradox' python library
2017-10-07 18:15:16,709 [11894] INFO gnocchi.cli: Joined coordination group: gnocchi-processing
2017-10-07 18:15:16,716 [11897] INFO gnocchi.cli: Joined coordination group: gnocchi-processing
2017-10-07 18:15:16,718 [11896] INFO gnocchi.cli: Joined coordination group: gnocchi-processing
2017-10-07 18:15:16,722 [11893] INFO gnocchi.cli: Joined coordination group: gnocchi-processing
2017-10-07 18:15:46,386 [11901] ERROR gnocchi.cli: Unexpected error during pending measures reporting
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/gnocchi/cli.py", line 169, in _run_job
    report = self.incoming.measures_report(details=False)
  File "/usr/local/lib/python2.7/dist-packages/gnocchi/storage/incoming/_carbonara.py", line 112, in measures_report
    metrics, measures, full_details = self._build_report(details)
  File "/usr/local/lib/python2.7/dist-packages/gnocchi/storage/incoming/ceph.py", line 117, in _build_report
    i, marker=marker, limit=self.Q_LIMIT))
  File "/usr/local/lib/python2.7/dist-packages/gnocchi/storage/incoming/ceph.py", line 139, in _list_keys_to_process
    op, self.get_sack_name(sack), flag=self.OMAP_READ_FLAGS)
  File "cradox.pyx", line 445, in cradox.requires.wrapper.validate_func (cradox.c:4401)
  File "cradox.pyx", line 3060, in cradox.Ioctx.operate_read_op (cradox.c:38177)
TimedOut: Failed to operate read op for oid incoming128-0
2017-10-07 18:15:46,727 [11893] ERROR gnocchi.cli: Unexpected error processing assigned job
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/gnocchi/cli.py", line 253, in _run_job
    metrics = in_store.list_metric_with_measures_to_process(s)
  File "/usr/local/lib/python2.7/dist-packages/gnocchi/storage/incoming/ceph.py", line 161, in list_metric_with_measures_to_process
    sack, marker=marker, limit=self.Q_LIMIT))
  File "/usr/local/lib/python2.7/dist-packages/gnocchi/storage/incoming/ceph.py", line 139, in _list_keys_to_process
    op, self.get_sack_name(sack), flag=self.OMAP_READ_FLAGS)
  File "cradox.pyx", line 445, in cradox.requires.wrapper.validate_func (cradox.c:4401)
  File "cradox.pyx", line 3060, in cradox.Ioctx.operate_read_op (cradox.c:38177)
TimedOut: Failed to operate read op for oid incoming128-0

Question information

Language:
English Edit question
Status:
Expired
For:
Ubuntu Edit question
Assignee:
No assignee Edit question
Last query:
2017-10-23
Last reply:
2017-11-08
Launchpad Janitor (janitor) said : #1

This question was expired because it remained in the 'Open' state without activity for the last 15 days.

I still have the problem.

> On 23 ott 2017, at 11:28, Launchpad Janitor <email address hidden> wrote:
>
> Your question #659113 on Ubuntu changed:
> https://answers.launchpad.net/ubuntu/+question/659113
>
> Status: Open => Expired
>
> Launchpad Janitor expired the question:
> This question was expired because it remained in the 'Open' state
> without activity for the last 15 days.
>
> --
> If you're still having this problem, you can reopen your question either
> by replying to this email or by going to the following page and
> entering more information about your problem:
> https://answers.launchpad.net/ubuntu/+question/659113
>
> You received this question notification because you asked the question.

Launchpad Janitor (janitor) said : #3

This question was expired because it remained in the 'Open' state without activity for the last 15 days.