We are using the graphiteapp/graphite-statsd docker container on an amazon i3.large instance. We are trying to send about 10k metrics per minute from a handful of hosts. We noticed we were missing a lot of data points so I started investigating performance. I wasn't able to find documentation on carbon's stats but I am seeing about 5k "droppedCreates" per minute and "committedPoints" of about 150 per minute. Looking at the code it seems that we are hitting some performance issues, but its not clear to me what the bottleneck is. I tried setting MAX_UPDATES_PER_SECOND = inf but that didn't change the behavior (still getting lots of droppedCreates).
carbon.agents.*.cache.size never goes above 100.
carbon.agents.*.cache.overflow stays at 0.
carbon.agents.*.creates are at about 50 per minute.
I'm seeing barely any CPU or mem usage by the agents or the aggregator.