Cluster slow when one node i down

I have a 2 graphite nodes setup behind an elastic load balancer in AWS. local_settings.py is as follows:

MEMCACHE_HOSTS = ['<graphite-1>:11211', '<graphite-2>:11211']
DEFAULT_CACHE_DURATION = 60
DEFAULT_CACHE_POLICY = [(0, 60), # default is 60 seconds
                        (7200, 120), # >= 2 hour queries are cached 2 minutes
                        (21600, 180), # >= 6 hour queries are cached 3 minutes
                        (43200, 360), # >= 12 hour queries are cached 6 minutes
                        (86400, 600)] # >= 24 hour queries are cached 10 minutes

SECRET_KEY = '****************************'

DATABASES = {
    'default': {
        'NAME': '/opt/graphite/storage/graphite.db',
        'ENGINE': 'django.db.backends.sqlite3',
        'USER': '',
        'PASSWORD': '',
        'HOST': '',
        'PORT': ''
    }
}

CLUSTER_SERVERS=["<graphite-1>","<graphite-2>"]

When one node goes down the response times become significantly longer. I suspect it's because one graphite node is still looking for metrics on the other node, even though it is down.

Is there a way to make the response times faster when one node is down? Or do I need to re-think my setup perhaps?