CPU reach 100% and become unresponsive after more than 1024 connections

Asked by ZoeSu

Hi,

I am using a Amazon EC2 Linux image to run mosquitto server. (v1.1.3)

I did the following to increase # of connections.
ulimit -u 4096

Then I restarted the mosquitto server. Whenever the number of subscribed clients go over 1024, the mosquitto service starts using 100% CPU resources, and it appears to hang and become unresponsive.

To reproduce this issue, I use the following shell script and the mosquitto_sub tool to simulate the over 1000+ connections:
------------------------------
gen_name()
{
 if [ $1 -lt 10 ]
 then
  name="topic0000"$1 #topic00009
 elif [ $1 -lt 100 ]
 then
  name="topic000"$1 #topic00099
 elif [ $1 -lt 1000 ]
 then
  name="topic00"$1 #topic00999
 elif [ $1 -lt 10000 ]
 then
  name="topic0"$1 #topic09999
 else
  name="topic"$1 #topic99999
 fi
}

max=4000
i=1
while [ $i -le ${max} ]
do
 gen_name $i
 mosquitto_sub -h x.x.x.x -p 8883 --cafile ca.crt --cert clit.crt --key clit.key -t $name -k 10 &
 sleep 0.01
 let i=$i+1
done
------------------------------

Question information

Language:
English Edit question
Status:
Answered
For:
mosquitto Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Roger Light (roger.light) said :
#1

The problem is that your OS sets a limit on the number of connections. Once this has been exceeded, all future connections are refused. Each time a client attempts to connect it causes the accept code to be run however, so you see a lot of processor overhead. You should try increasing the limit using "ulimit -n" or the global limits in /etc/limits.conf.

On a slightly different note, your loop can be made simpler. I would do this:

for i in $(seq 1 ${max}); do
    ....
done

Can you help with this problem?

Provide an answer of your own, or ask ZoeSu for more information if necessary.

To post a message you must log in.