Getting internal error for graphite

Asked by rocky

Hi,

After installation of graphite getting Internal server error (error code : 500) while hitting from the browser. Could you please check and provide your inputs.

Below log path : /opt/graphite/storage/log/webapp

file : error.log

[Tue Sep 26 01:28:41 2017] [error] [client 10.2.200.22] mod_wsgi (pid=21224): Target WSGI script '/opt/graphite/conf/graphite.wsgi' cannot be loaded as Python module.
[Tue Sep 26 01:28:41 2017] [error] [client 10.2.200.22] mod_wsgi (pid=21224): Exception occurred processing WSGI script '/opt/graphite/conf/graphite.wsgi'.
[Tue Sep 26 01:28:41 2017] [error] [client 10.2.200.22] Traceback (most recent call last):
[Tue Sep 26 01:28:41 2017] [error] [client 10.2.200.22] File "/opt/graphite/conf/graphite.wsgi", line 4, in <module>
[Tue Sep 26 01:28:41 2017] [error] [client 10.2.200.22] from graphite.wsgi import application
[Tue Sep 26 01:28:41 2017] [error] [client 10.2.200.22] File "/opt/graphite/webapp/graphite/wsgi.py", line 7, in <module>
[Tue Sep 26 01:28:41 2017] [error] [client 10.2.200.22] from django.utils.importlib import import_module
[Tue Sep 26 01:28:41 2017] [error] [client 10.2.200.22] ImportError: No module named django.utils.importlib

Thank you in advance.

Question information

Language:
English Edit question
Status:
Solved
For:
Graphite Edit question
Assignee:
No assignee Edit question
Solved by:
rocky
Solved:
Last query:
Last reply:
Revision history for this message
Denis Zhdanov (deniszhdanov) said :
#1

What version of Graphite you trying to install? and what's Django version do you have?

Revision history for this message
Denis Zhdanov (deniszhdanov) said :
#2

PS: For Graphite 1.0.x wsgi can be very simple - https://github.com/graphite-project/graphite-web/blob/master/conf/graphite.wsgi.example, .e.g.

import sys
sys.path.append('/opt/graphite/webapp')
from graphite.wsgi import application

Revision history for this message
rocky (rocky.munich1) said :
#3

Hi Denis

Please find the details.

[root@XX ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 6.9 (Santiago)
[root@XX ~]#

[root@XX ~]# python -V
Python 2.7.3
[root@XX ~]#

>>> import django
>>> django.VERSION
(1, 9, 9, 'final', 0)
>>>

graphite_web-1.1.0

Revision history for this message
rocky (rocky.munich1) said :
#4

Hi

please find the more information.

[root@XX conf.d]# cat /opt/graphite/conf/graphite.wsgi
import sys
sys.path.append('/opt/graphite/webapp')

from graphite.wsgi import application
[root@XX conf.d]#

[root@XX conf.d]# cat /opt/graphite/webapp/graphite/wsgi.py
import os
import sys

try:
    from importlib import import_module
except ImportError:
    from django.utils.importlib import import_module

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'graphite.settings') # noqa

from django.conf import settings
from django.core.wsgi import get_wsgi_application

application = get_wsgi_application()

try:
    import whitenoise
except ImportError:
    whitenoise = False
else:
    # WhiteNoise < 2.0.1 does not support Python 2.6
    if sys.version_info[:2] < (2, 7):
        whitenoise_version = tuple(map(
                int, getattr(whitenoise, '__version__', '0').split('.')))
        if whitenoise_version < (2, 0, 1):
            whitenoise = False

if whitenoise:
    from whitenoise.django import DjangoWhiteNoise
    application = DjangoWhiteNoise(application)
    prefix = "/".join((settings.URL_PREFIX.strip('/'), 'static'))
    for directory in settings.STATICFILES_DIRS:
        application.add_files(directory, prefix=prefix)
    for app_path in settings.INSTALLED_APPS:
        module = import_module(app_path)
        directory = os.path.join(os.path.dirname(module.__file__), 'static')
        if os.path.isdir(directory):
            application.add_files(directory, prefix=prefix)
[root@XX conf.d]#

[root@XX conf.d]# cat /etc/httpd/conf.d/graphite.conf
# Enable virtualhosts, perhaps by adding this to your server's config somewhere,
# probably the main httpd.conf
# NameVirtualHost *:80

# This line also needs to be in your server's config.
# LoadModule wsgi_module modules/mod_wsgi.so

# You need to manually edit this file to fit your needs.
# This configuration assumes the default installation prefix
# of /opt/graphite/, if you installed graphite somewhere else
# you will need to change all the occurrences of /opt/graphite/
# in this file to your chosen install location.

<IfModule !wsgi_module.c>
    LoadModule wsgi_module modules/mod_wsgi.so
</IfModule>

# XXX You need to set this up!
# Read http://code.google.com/p/modwsgi/wiki/ConfigurationDirectives#WSGISocketPrefix
# For example, create a directory /var/run/wsgi and use that.
WSGISocketPrefix run/wsgi

<VirtualHost *:8080>
        ServerName XX.XX.XX.XX
        DocumentRoot "/opt/graphite/webapp"
        ErrorLog /opt/graphite/storage/log/webapp/error.log
        CustomLog /opt/graphite/storage/log/webapp/access.log common
        Header set Access-Control-Allow-Origin "*"

        # I've found that an equal number of processes & threads tends
        # to show the best performance for Graphite (ymmv).
        WSGIDaemonProcess graphite processes=5 threads=5 display-name='%{GROUP}' inactivity-timeout=120
        WSGIProcessGroup graphite
        WSGIApplicationGroup %{GLOBAL}
        WSGIImportScript /opt/graphite/conf/graphite.wsgi process-group=graphite application-group=%{GLOBAL}

        # XXX You will need to create this file! There is a graphite.wsgi.example
        # file in this directory that you can safely use, just copy it to graphite.wgsi
        WSGIScriptAlias / /opt/graphite/conf/graphite.wsgi

        # XXX To serve static files, either:
        # * Install the whitenoise Python package (pip install whitenoise)
        # * Collect static files in a directory by running:
        # django-admin.py collectstatic --noinput --settings=graphite.settings
        # And set an alias to serve static files with Apache:
        Alias /static/ /opt/graphite/static/
        <Directory /opt/graphite/static/>
                        Order deny,allow
                        Allow from all
        </Directory>

        ########################
        # URL-prefixed install #
        ########################
        # If using URL_PREFIX in local_settings for URL-prefixed install (that is not located at "/"))
        # your WSGIScriptAlias line should look like the following (e.g. URL_PREFX="/graphite"

        # WSGIScriptAlias /graphite /srv/graphite-web/conf/graphite.wsgi/graphite
        # Alias /graphite/static /opt/graphite/webapp/content
        # <Location "/graphite/static/">
        # SetHandler None
        # </Location>

        # XXX In order for the django admin site media to work you
        # must change @DJANGO_ROOT@ to be the path to your django
        # installation, which is probably something like:
        # /usr/lib/python2.6/site-packages/django
        Alias /media/ "@DJANGO_ROOT@/contrib/admin/media/"

        # The graphite.wsgi file has to be accessible by apache. It won't
        # be visible to clients because of the DocumentRoot though.
        <Directory /opt/graphite/conf/>
                <IfVersion < 2.4>
                        Order deny,allow
                        Allow from all
                </IfVersion>
                <IfVersion >= 2.4>
                        Require all granted
                </IfVersion>
        </Directory>

</VirtualHost>
[root@XX conf.d]#

Revision history for this message
Denis Zhdanov (deniszhdanov) said :
#5

Just use simple wsgi file which I posted above.

Revision history for this message
rocky (rocky.munich1) said :
#6

Thank you for your response.

We are already using same same.

Revision history for this message
Denis Zhdanov (deniszhdanov) said :
#7

Ah, true.
django.utils.importlib was deprecated in django 1.9, but looks like your code still using old version oj django. try to remove all versions of django except 1.9

Revision history for this message
rocky (rocky.munich1) said :
#8

Thank you for response.

I can see only one Django==1.9.9 in the system.

Could you please help me to find out other version in the system.

Revision history for this message
rocky (rocky.munich1) said :
#9

Could you please some one help on this.. really appreciated..

Revision history for this message
Denis Zhdanov (deniszhdanov) said :
#10

Remove old fine (/opt/graphite/webapp/graphite/wsgi.py) and try again.

Revision history for this message
rocky (rocky.munich1) said :
#12

Hi Deniz

After removing the file same repose.

[root@XX webapp]# rm /opt/graphite/webapp/graphite/wsgi.py
rm: remove regular file `/opt/graphite/webapp/graphite/wsgi.py'? y
[root@icingamonitoringdr webapp]#

[root@XX webapp]# ls -l "/opt/graphite/webapp/graphite/wsgi.py"
ls: cannot access /opt/graphite/webapp/graphite/wsgi.py: No such file or directory
[root@XX webapp]#

Still getting error

:58:55 2017] [error] [client 10.2.201.115] mod_wsgi (pid=11944): Exception occurred processing WSGI script '/opt/graphite/conf/graphite.wsgi'.
[Thu Sep 28 02:58:55 2017] [error] [client 10.2.201.115] Traceback (most recent call last):
[Thu Sep 28 02:58:55 2017] [error] [client 10.2.201.115] File "/opt/graphite/conf/graphite.wsgi", line 4, in <module>
[Thu Sep 28 02:58:55 2017] [error] [client 10.2.201.115] from graphite.wsgi import application
[Thu Sep 28 02:58:55 2017] [error] [client 10.2.201.115] File "/opt/graphite/webapp/graphite/wsgi.py", line 7, in <module>
[Thu Sep 28 02:58:55 2017] [error] [client 10.2.201.115] ImportError: No module named django.utils.importlib

Revision history for this message
Denis Zhdanov (deniszhdanov) said :
#13

How you can get error in line 7 of non-existing File "/opt/graphite/webapp/graphite/wsgi.py" ?

Revision history for this message
rocky (rocky.munich1) said :
#14

Seems to be something else having the issue.

Revision history for this message
rocky (rocky.munich1) said :
#15

after rebooting the server.. it is saying site can't be reached.

Revision history for this message
rocky (rocky.munich1) said :
#16

Hi Denis,

Could you please cross check the file "/etc/httpd/conf.d/graphite.conf" mentioned above.

Revision history for this message
Denis Zhdanov (deniszhdanov) said :
#17

Looks sane. You need to remove 'Alias /media/ "@DJANGO_ROOT@/contrib/admin/media/"' line, but it should not bring any troubles.

Revision history for this message
rocky (rocky.munich1) said :
#18

We removed the file and tried..
still getting same issue in the error log

[Tue Sep 26 01:28:41 2017] [error] [client 10.2.200.22] File "/opt/graphite/conf/graphite.wsgi", line 4, in <module>
[Tue Sep 26 01:28:41 2017] [error] [client 10.2.200.22] from graphite.wsgi import application
[Tue Sep 26 01:28:41 2017] [error] [client 10.2.200.22] File "/opt/graphite/webapp/graphite/wsgi.py", line 7, in <module>
[Tue Sep 26 01:28:41 2017] [error] [client 10.2.200.22] from django.utils.importlib import import_module
[Tue Sep 26 01:28:41 2017] [error] [client 10.2.200.22] ImportError: No module named django.utils.importlib

Revision history for this message
Denis Zhdanov (deniszhdanov) said :
#19

Sorry, rocky, but this lines are old - from September 26. I'm really doubt that non-existing file can give you this error, especially after server reboot. Please check fresh errors.

Revision history for this message
rocky (rocky.munich1) said :
#20

ok sure...
 Same error.. that is the reason copied for the above line..
Let me share the fresh error log.

[Fri Sep 29 00:11:06 2017] [error] [client 10.2.200.191] mod_wsgi (pid=17662): Target WSGI script '/opt/graphite/conf/graphite.wsgi' cannot be loaded as Python module.
[Fri Sep 29 00:11:06 2017] [error] [client 10.2.200.191] mod_wsgi (pid=17662): Exception occurred processing WSGI script '/opt/graphite/conf/graphite.wsgi'.
[Fri Sep 29 00:11:06 2017] [error] [client 10.2.200.191] Traceback (most recent call last):
[Fri Sep 29 00:11:06 2017] [error] [client 10.2.200.191] File "/opt/graphite/conf/graphite.wsgi", line 4, in <module>
[Fri Sep 29 00:11:06 2017] [error] [client 10.2.200.191] from graphite.wsgi import application
[Fri Sep 29 00:11:06 2017] [error] [client 10.2.200.191] File "/opt/graphite/webapp/graphite/wsgi.py", line 7, in <module>
[Fri Sep 29 00:11:06 2017] [error] [client 10.2.200.191] from django.utils.importlib import import_module
[Fri Sep 29 00:11:06 2017] [error] [client 10.2.200.191] ImportError: No module named django.utils.importlib

Revision history for this message
Denis Zhdanov (deniszhdanov) said :
#21

Sorry, rocky, I ran out of ideas. You should remove file /opt/graphite/webapp/graphite/wsgi.py and restart webserver - error should gone. If it's not - probably you're checking wrong server, or something else is seriously wrong.

Revision history for this message
rocky (rocky.munich1) said :
#22

Thank you for response.

Found something .. let me share with you..

[root@XX ~]# ldd /usr/lib64/httpd/modules/mod_wsgi.so
        linux-vdso.so.1 => (0x00007ffd08dcb000)
        libpython2.6.so.1.0 => /usr/lib64/libpython2.6.so.1.0 (0x00007febc266b000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007febc244e000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007febc2249000)
        libutil.so.1 => /lib64/libutil.so.1 (0x00007febc2046000)
        libm.so.6 => /lib64/libm.so.6 (0x00007febc1dc2000)
        libc.so.6 => /lib64/libc.so.6 (0x00007febc1a2d000)
        /lib64/ld-linux-x86-64.so.2 (0x0000003adba00000)
[root@XX ~]#

[root@XX ~]# find / -name libpython2.7*
/usr/local/lib/python2.7/config/libpython2.7.a
/usr/local/lib/libpython2.7.a
/usr/local/bin/Python-2.7.3/libpython2.7.a
[root@XX ~]#

How to load the module with python version 2.7 ?

Revision history for this message
Denis Zhdanov (deniszhdanov) said :
#23

You should install proper mod_wsgi module, which build against python 2.7
It depends where do you get python 2.7
If you installed that from 3rd party repo - it's better to pick up mod_wsgi package also there.
if you install python from sources - then you need to build mod_wsgi also from sources, as described for example here - https://blog.webhostpython.com/how-to-install-python-2-7-with-mod_wsgi-on-a-centos-6-vps-or-server-with-cpanel/

Revision history for this message
rocky (rocky.munich1) said :
#24

Thank you for the update..

Issue resolved after updating the module mod_wsgi.so. Please find the reference links and commands

https://serverfault.com/questions/453977/configuring-mod-wsgi-to-use-python-2-7
yum install httpd-devel

https://stackoverflow.com/questions/20842732/libpython2-7-so-1-0-cannot-open-shared-object-file-no-such-file-or-directory

/etc/ld.so.conf --> Need to update with location
locate libpython2.7.so.1.0
ldconfig

cd /usr/local/bin/Python-2.7.3
./configure –-enable-shared CFLAGS=-fPIC
Make
Make install

Cd /root/mod_wsgi-3.4
./configure –-with-python=/usr/local/bin/Python-2.7.3/python
Make
Make install

Service httpd restart

Ldd /usr/lib64/httpd/modules/mod_wsgi.so

Netstat –ltnp | grep 80