Graphite

graphite is giving a MemoryError traceback on a server with 10G of free memory

Asked by Jeff Schroeder on 2010-11-10

So after a friend (who works for a large ISP) who uses graphite convinced me it was amazing, I set it up. I followed the install doc and got graphite running. Graphite is running with the run-devel-server script, carbon with --debug, and the example client injects metrics every 30 seconds.

Whenever logging into graphite, the main graph is broken and I'm unsure why. In the runserver output, I saw things like:

[09/Nov/2010 17:08:36] "GET /render HTTP/1.1" 500 77007
[09/Nov/2010 17:08:36] "GET /render/?width=586&height=303&_salt=1289351315.838 HTTP/1.1" 500 78074

Lucky for me, I've got some django dev experience and a lot of django debugging experience. I mudged settings.py to point at the django-sentry[1] server we already have setup for many of our other django apps. It gave me this traceback:

Request Method: GET
Request URL: http://labserver.int:8080/render/?width=586&height=303&_salt=1289351320.119
Python Version:

Traceback:
File "/usr/lib/python2.5/site-packages/django/core/handlers/base.py" in get_response
  100. response = callback(request, *callback_args, **callback_kwargs)
File "/opt/graphite/webapp/graphite/render/views.py" in renderView
  128. image = doImageRender(requestOptions['graphClass'], graphOptions)
File "/opt/graphite/webapp/graphite/render/views.py" in doImageRender
  283. img = graphClass(**graphOptions)
File "/opt/graphite/webapp/graphite/render/glyph.py" in __init__
  140. self.drawText("No Data", x, y, align='center')
File "/opt/graphite/webapp/graphite/render/glyph.py" in drawText
  200. extents = self.getExtents(text)
File "/opt/graphite/webapp/graphite/render/glyph.py" in getExtents
  172. F = self.ctx.font_extents()

Exception Type: MemoryError at /render/?width=586&height=303&_salt=1289351320.119
Exception Value:

<email address hidden>: graphite $ free -m
             total used free shared buffers cached
Mem: 11906 1464 10441 0 194 833
-/+ buffers/cache: 437 11469
Swap: 8189 0 8189

What am I doing wrong to cause this? Thanks!

Question information

Language:
English Edit question
Status:
Solved
For:
Graphite Edit question
Assignee:
No assignee Edit question
Solved by:
Jeff Schroeder
Solved:
2010-11-10
Last query:
2010-11-10
Last reply:
2010-11-10
Ilya Zakreuski (zlj) said : #1

Hi Jeff,

please check this question: https://answers.launchpad.net/graphite/+question/127623, it looks pretty the same.

Also I would appreciate any feedback on a packages on which I've been working some time ago: https://launchpad.net/~zlj/+archive/graphite-dev. I hope to have most of installation issues being resolved in a packaged version.

Cheers!

--
Ilya Zakreuski

Jeff Schroeder (sejeff) said : #2

@Ilya: Installing bitmap-fonts did indeed fix the problem. I'm using a Fedora Core 10 server and ppas dont work so well :)

I did try out the rpm packages that exist for 0.9.5, but wanted to test 0.9.6. So I downloaded the 0.9.5 src.rpm, cracked open the spec, and beat on it for awhile. It looked like the upstream tarball had been changed in the src rpms I found ie: carbon-0.9.6 was moved to python-carbon-0.9.6, and I think some of the files might have been changed. After playing with the spec file and probably 20 mock builds with packaging errors, I gave up as installing from source using setup.py install works.

Thanks!