mosquitto

assert error when stop server

Asked by jinlove

I always came cross such a error in the log when i try to stop mosquitto with "/etc/init.d/mosquitto stop" :

"mqtt3_context_cleanup: Assertion `context->listener->client_count >= 0' failed"

I launch several mosquitto instances with different ports and different persistent files, and the error always(not every time) happens on every instance after 3-4 days since last start, and the server just can't start again until i manually remove the persistence file.

version : 1.2.3
deploy on wheezy

thanks.

Question information

Language:
English Edit question
Status:
Answered
For:
mosquitto Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Roger Light (roger.light) said : 		#1

Can you provide any more details about your config please?

On Tue, Jun 3, 2014 at 9:56 AM, jinlove
<email address hidden> wrote:
> New question #249688 on mosquitto:
> https://answers.launchpad.net/mosquitto/+question/249688
>
> I always came cross such a error in the log when i try to stop mosquitto with "/etc/init.d/mosquitto stop" :
>
> "mqtt3_context_cleanup: Assertion `context->listener->client_count >= 0' failed"
>
> I launch several mosquitto instances with different ports and different persistent files, and the error always(not every time) happens on every instance after 3-4 days since last start, and the server just can't start again until i manually remove the persistence file.
>
> version : 1.2.3
> deploy on wheezy
>
> thanks.
>
> --
> You received this question notification because you are a member of
> Mosquitto PPA, which is an answer contact for mosquitto.

Revision history for this message
jinlove (wonderfulseason) said : 		#2

very glad to receive your reply.

Here is my explicit config per instances, and the port is replaced on
launch.

pid_file /var/run/mosquitto-1883.pid
port 1883
persistence true
persistence_file mosquitto-1883.db
persistence_location /var/lib/mosquitto/
log_dest stdout
allow_anonymous false
password_file /etc/mosquitto/pwd.conf
acl_file /etc/mosquitto/acl.conf

other option are commented by default.

Here is my /etc/init.d/mosquitto.

DAEMON=/usr/sbin/mosquitto
LOG_PATH=/var/log/mosquitto.log

# /etc/init.d/mosquitto: start and stop the mosquitto MQTT message broker

test -x ${DAEMON} || exit 0

umask 022

. /lib/lsb/init-functions

# Are we running from init?
run_by_init() {
    ([ "$previous" ] && [ "$runlevel" ]) || [ "$runlevel" = S ]
}

export PATH="${PATH:+$PATH:}/usr/sbin:/sbin"

ulimit -n 65536

for port in $(seq 1860 1899); do

BASE_CONF=/etc/mosquitto/mosquitto.conf
PORT_CONF=/etc/mosquitto/mosquitto-$port.conf
if [ "$BASE_CONF" -nt "$PORT_CONF" ]; then
    echo "Updating $PORT_CONF"
    sed "s/@port/$port/g" "$BASE_CONF" > "$PORT_CONF"
fi

PIDFILE=/var/run/mosquitto-$port.pid

case "$1" in
  start)
        if init_is_upstart; then
            exit 1
        fi
        log_daemon_msg "Starting network daemon:" "mosquitto-$port"
        if start-stop-daemon --start --quiet --oknodo --background
--make-pidfile --pidfile ${PIDFILE} --no-close --exec ${DAEMON} -- -c
"$PORT_CONF" >> $LOG_PATH 2>&1 ; then
            log_end_msg 0
        else
            log_end_msg 1
        fi
        ;;
  stop)
        if init_is_upstart; then
            exit 0
        fi
        log_daemon_msg "Stopping network daemon:" "mosquitto-$port"
        if start-stop-daemon --stop --quiet --oknodo --pidfile ${PIDFILE};
then
            log_end_msg 0
            rm -f ${PIDFILE}
        else
            log_end_msg 1
        fi
        ;;
...

Thanks very much.

2014-06-03 17:17 GMT+08:00 Roger Light <<email address hidden>
>:

> Your question #249688 on mosquitto changed:
> https://answers.launchpad.net/mosquitto/+question/249688
>
> Status: Open => Answered
>
> Roger Light proposed the following answer:
> Can you provide any more details about your config please?
>
> On Tue, Jun 3, 2014 at 9:56 AM, jinlove
> <email address hidden> wrote:
> > New question #249688 on mosquitto:
> > https://answers.launchpad.net/mosquitto/+question/249688
> >
> > I always came cross such a error in the log when i try to stop mosquitto
> with "/etc/init.d/mosquitto stop" :
> >
> > "mqtt3_context_cleanup: Assertion `context->listener->client_count >= 0'
> failed"
> >
> > I launch several mosquitto instances with different ports and different
> persistent files, and the error always(not every time) happens on every
> instance after 3-4 days since last start, and the server just can't start
> again until i manually remove the persistence file.
> >
> > version : 1.2.3
> > deploy on wheezy
> >
> > thanks.
> >
> > --
> > You received this question notification because you are a member of
> > Mosquitto PPA, which is an answer contact for mosquitto.
>
> --
> If this answers your question, please go to the following page to let us
> know that it is solved:
>
> https://answers.launchpad.net/mosquitto/+question/249688/+confirm?answer_id=0
>
> If you still need help, you can reply to this email or go to the
> following page to enter your feedback:
> https://answers.launchpad.net/mosquitto/+question/249688
>
> You received this question notification because you asked the question.
>

--
Lawrence

Revision history for this message
Roger Light (roger.light) said : 		#3

That's great, could you also give me a few details about how your
clients connect? I'm interested mostly in whether you use
clean_session or not, but anything you think might be relevant would
be appreciated.

On Tue, Jun 3, 2014 at 10:51 AM, jinlove
<email address hidden> wrote:
> Question #249688 on mosquitto changed:
> https://answers.launchpad.net/mosquitto/+question/249688
>
> Status: Answered => Open
>
> jinlove is still having a problem:
> very glad to receive your reply.
>
> Here is my explicit config per instances, and the port is replaced on
> launch.
>
> pid_file /var/run/mosquitto-1883.pid
> port 1883
> persistence true
> persistence_file mosquitto-1883.db
> persistence_location /var/lib/mosquitto/
> log_dest stdout
> allow_anonymous false
> password_file /etc/mosquitto/pwd.conf
> acl_file /etc/mosquitto/acl.conf
>
> other option are commented by default.
>
> Here is my /etc/init.d/mosquitto.
>
> DAEMON=/usr/sbin/mosquitto
> LOG_PATH=/var/log/mosquitto.log
>
> # /etc/init.d/mosquitto: start and stop the mosquitto MQTT message
> broker
>
> test -x ${DAEMON} || exit 0
>
> umask 022
>
> . /lib/lsb/init-functions
>
> # Are we running from init?
> run_by_init() {
> ([ "$previous" ] && [ "$runlevel" ]) || [ "$runlevel" = S ]
> }
>
> export PATH="${PATH:+$PATH:}/usr/sbin:/sbin"
>
> ulimit -n 65536
>
> for port in $(seq 1860 1899); do
>
> BASE_CONF=/etc/mosquitto/mosquitto.conf
> PORT_CONF=/etc/mosquitto/mosquitto-$port.conf
> if [ "$BASE_CONF" -nt "$PORT_CONF" ]; then
> echo "Updating $PORT_CONF"
> sed "s/@port/$port/g" "$BASE_CONF" > "$PORT_CONF"
> fi
>
> PIDFILE=/var/run/mosquitto-$port.pid
>
> case "$1" in
> start)
> if init_is_upstart; then
> exit 1
> fi
> log_daemon_msg "Starting network daemon:" "mosquitto-$port"
> if start-stop-daemon --start --quiet --oknodo --background
> --make-pidfile --pidfile ${PIDFILE} --no-close --exec ${DAEMON} -- -c
> "$PORT_CONF" >> $LOG_PATH 2>&1 ; then
> log_end_msg 0
> else
> log_end_msg 1
> fi
> ;;
> stop)
> if init_is_upstart; then
> exit 0
> fi
> log_daemon_msg "Stopping network daemon:" "mosquitto-$port"
> if start-stop-daemon --stop --quiet --oknodo --pidfile ${PIDFILE};
> then
> log_end_msg 0
> rm -f ${PIDFILE}
> else
> log_end_msg 1
> fi
> ;;
> ...
>
> Thanks very much.
>
>
>
>
>
> 2014-06-03 17:17 GMT+08:00 Roger Light <<email address hidden>
>>:
>
>> Your question #249688 on mosquitto changed:
>> https://answers.launchpad.net/mosquitto/+question/249688
>>
>> Status: Open => Answered
>>
>> Roger Light proposed the following answer:
>> Can you provide any more details about your config please?
>>
>> On Tue, Jun 3, 2014 at 9:56 AM, jinlove
>> <email address hidden> wrote:
>> > New question #249688 on mosquitto:
>> > https://answers.launchpad.net/mosquitto/+question/249688
>> >
>> > I always came cross such a error in the log when i try to stop mosquitto
>> with "/etc/init.d/mosquitto stop" :
>> >
>> > "mqtt3_context_cleanup: Assertion `context->listener->client_count >= 0'
>> failed"
>> >
>> > I launch several mosquitto instances with different ports and different
>> persistent files, and the error always(not every time) happens on every
>> instance after 3-4 days since last start, and the server just can't start
>> again until i manually remove the persistence file.
>> >
>> > version : 1.2.3
>> > deploy on wheezy
>> >
>> > thanks.
>> >
>> > --
>> > You received this question notification because you are a member of
>> > Mosquitto PPA, which is an answer contact for mosquitto.
>>
>> --
>> If this answers your question, please go to the following page to let us
>> know that it is solved:
>>
>> https://answers.launchpad.net/mosquitto/+question/249688/+confirm?answer_id=0
>>
>> If you still need help, you can reply to this email or go to the
>> following page to enter your feedback:
>> https://answers.launchpad.net/mosquitto/+question/249688
>>
>> You received this question notification because you asked the question.
>>
>
>
> --
> Lawrence
>
> You received this question notification because you are a member of
> Mosquitto PPA, which is an answer contact for mosquitto.

Revision history for this message
jinlove (wonderfulseason) said : 		#4

I use Paho as client and clean session is set to false.

There are 10k connection per instance, and each client subscribe two topics.

BTW, not every time will it throw assert error when i stop instance, but
with very high probability if it is running for several days or perhaps too
many subscriptions (too many unactive clients)

Thanks.

2014-06-03 19:02 GMT+08:00 Roger Light <<email address hidden>
>:

> Your question #249688 on mosquitto changed:
> https://answers.launchpad.net/mosquitto/+question/249688
>
> Status: Open => Answered
>
> Roger Light proposed the following answer:
> That's great, could you also give me a few details about how your
> clients connect? I'm interested mostly in whether you use
> clean_session or not, but anything you think might be relevant would
> be appreciated.
>
> On Tue, Jun 3, 2014 at 10:51 AM, jinlove
> <email address hidden> wrote:
> > Question #249688 on mosquitto changed:
> > https://answers.launchpad.net/mosquitto/+question/249688
> >
> > Status: Answered => Open
> >
> > jinlove is still having a problem:
> > very glad to receive your reply.
> >
> > Here is my explicit config per instances, and the port is replaced on
> > launch.
> >
> > pid_file /var/run/mosquitto-1883.pid
> > port 1883
> > persistence true
> > persistence_file mosquitto-1883.db
> > persistence_location /var/lib/mosquitto/
> > log_dest stdout
> > allow_anonymous false
> > password_file /etc/mosquitto/pwd.conf
> > acl_file /etc/mosquitto/acl.conf
> >
> > other option are commented by default.
> >
> > Here is my /etc/init.d/mosquitto.
> >
> > DAEMON=/usr/sbin/mosquitto
> > LOG_PATH=/var/log/mosquitto.log
> >
> > # /etc/init.d/mosquitto: start and stop the mosquitto MQTT message
> > broker
> >
> > test -x ${DAEMON} || exit 0
> >
> > umask 022
> >
> > . /lib/lsb/init-functions
> >
> > # Are we running from init?
> > run_by_init() {
> > ([ "$previous" ] && [ "$runlevel" ]) || [ "$runlevel" = S ]
> > }
> >
> > export PATH="${PATH:+$PATH:}/usr/sbin:/sbin"
> >
> > ulimit -n 65536
> >
> > for port in $(seq 1860 1899); do
> >
> > BASE_CONF=/etc/mosquitto/mosquitto.conf
> > PORT_CONF=/etc/mosquitto/mosquitto-$port.conf
> > if [ "$BASE_CONF" -nt "$PORT_CONF" ]; then
> > echo "Updating $PORT_CONF"
> > sed "s/@port/$port/g" "$BASE_CONF" > "$PORT_CONF"
> > fi
> >
> > PIDFILE=/var/run/mosquitto-$port.pid
> >
> > case "$1" in
> > start)
> > if init_is_upstart; then
> > exit 1
> > fi
> > log_daemon_msg "Starting network daemon:" "mosquitto-$port"
> > if start-stop-daemon --start --quiet --oknodo --background
> > --make-pidfile --pidfile ${PIDFILE} --no-close --exec ${DAEMON} -- -c
> > "$PORT_CONF" >> $LOG_PATH 2>&1 ; then
> > log_end_msg 0
> > else
> > log_end_msg 1
> > fi
> > ;;
> > stop)
> > if init_is_upstart; then
> > exit 0
> > fi
> > log_daemon_msg "Stopping network daemon:" "mosquitto-$port"
> > if start-stop-daemon --stop --quiet --oknodo --pidfile
> ${PIDFILE};
> > then
> > log_end_msg 0
> > rm -f ${PIDFILE}
> > else
> > log_end_msg 1
> > fi
> > ;;
> > ...
> >
> > Thanks very much.
> >
> >
> >
> >
> >
> > 2014-06-03 17:17 GMT+08:00 Roger Light <
> <email address hidden>
> >>:
> >
> >> Your question #249688 on mosquitto changed:
> >> https://answers.launchpad.net/mosquitto/+question/249688
> >>
> >> Status: Open => Answered
> >>
> >> Roger Light proposed the following answer:
> >> Can you provide any more details about your config please?
> >>
> >> On Tue, Jun 3, 2014 at 9:56 AM, jinlove
> >> <email address hidden> wrote:
> >> > New question #249688 on mosquitto:
> >> > https://answers.launchpad.net/mosquitto/+question/249688
> >> >
> >> > I always came cross such a error in the log when i try to stop
> mosquitto
> >> with "/etc/init.d/mosquitto stop" :
> >> >
> >> > "mqtt3_context_cleanup: Assertion `context->listener->client_count >=
> 0'
> >> failed"
> >> >
> >> > I launch several mosquitto instances with different ports and
> different
> >> persistent files, and the error always(not every time) happens on every
> >> instance after 3-4 days since last start, and the server just can't
> start
> >> again until i manually remove the persistence file.
> >> >
> >> > version : 1.2.3
> >> > deploy on wheezy
> >> >
> >> > thanks.
> >> >
> >> > --
> >> > You received this question notification because you are a member of
> >> > Mosquitto PPA, which is an answer contact for mosquitto.
> >>
> >> --
> >> If this answers your question, please go to the following page to let us
> >> know that it is solved:
> >>
> >>
> https://answers.launchpad.net/mosquitto/+question/249688/+confirm?answer_id=0
> >>
> >> If you still need help, you can reply to this email or go to the
> >> following page to enter your feedback:
> >> https://answers.launchpad.net/mosquitto/+question/249688
> >>
> >> You received this question notification because you asked the question.
> >>
> >
> >
> > --
> > Lawrence
> >
> > You received this question notification because you are a member of
> > Mosquitto PPA, which is an answer contact for mosquitto.
>
> --
> If this answers your question, please go to the following page to let us
> know that it is solved:
>
> https://answers.launchpad.net/mosquitto/+question/249688/+confirm?answer_id=2
>
> If you still need help, you can reply to this email or go to the
> following page to enter your feedback:
> https://answers.launchpad.net/mosquitto/+question/249688
>
> You received this question notification because you asked the question.
>

--
Lawrence

Revision history for this message
Roger Light (roger.light) said : 		#5

I've made some changes that should hopefully help with this. It's
probably not that critical actually though, this assert happens quite
late on in the deinitialisation process.

On Tue, Jun 3, 2014 at 1:02 PM, jinlove
<email address hidden> wrote:
> Question #249688 on mosquitto changed:
> https://answers.launchpad.net/mosquitto/+question/249688
>
> Status: Answered => Open
>
> jinlove is still having a problem:
> I use Paho as client and clean session is set to false.
>
> There are 10k connection per instance, and each client subscribe two
> topics.
>
> BTW, not every time will it throw assert error when i stop instance, but
> with very high probability if it is running for several days or perhaps too
> many subscriptions (too many unactive clients)
>
> Thanks.
>
>
> 2014-06-03 19:02 GMT+08:00 Roger Light <<email address hidden>
>>:
>
>> Your question #249688 on mosquitto changed:
>> https://answers.launchpad.net/mosquitto/+question/249688
>>
>> Status: Open => Answered
>>
>> Roger Light proposed the following answer:
>> That's great, could you also give me a few details about how your
>> clients connect? I'm interested mostly in whether you use
>> clean_session or not, but anything you think might be relevant would
>> be appreciated.
>>
>> On Tue, Jun 3, 2014 at 10:51 AM, jinlove
>> <email address hidden> wrote:
>> > Question #249688 on mosquitto changed:
>> > https://answers.launchpad.net/mosquitto/+question/249688
>> >
>> > Status: Answered => Open
>> >
>> > jinlove is still having a problem:
>> > very glad to receive your reply.
>> >
>> > Here is my explicit config per instances, and the port is replaced on
>> > launch.
>> >
>> > pid_file /var/run/mosquitto-1883.pid
>> > port 1883
>> > persistence true
>> > persistence_file mosquitto-1883.db
>> > persistence_location /var/lib/mosquitto/
>> > log_dest stdout
>> > allow_anonymous false
>> > password_file /etc/mosquitto/pwd.conf
>> > acl_file /etc/mosquitto/acl.conf
>> >
>> > other option are commented by default.
>> >
>> > Here is my /etc/init.d/mosquitto.
>> >
>> > DAEMON=/usr/sbin/mosquitto
>> > LOG_PATH=/var/log/mosquitto.log
>> >
>> > # /etc/init.d/mosquitto: start and stop the mosquitto MQTT message
>> > broker
>> >
>> > test -x ${DAEMON} || exit 0
>> >
>> > umask 022
>> >
>> > . /lib/lsb/init-functions
>> >
>> > # Are we running from init?
>> > run_by_init() {
>> > ([ "$previous" ] && [ "$runlevel" ]) || [ "$runlevel" = S ]
>> > }
>> >
>> > export PATH="${PATH:+$PATH:}/usr/sbin:/sbin"
>> >
>> > ulimit -n 65536
>> >
>> > for port in $(seq 1860 1899); do
>> >
>> > BASE_CONF=/etc/mosquitto/mosquitto.conf
>> > PORT_CONF=/etc/mosquitto/mosquitto-$port.conf
>> > if [ "$BASE_CONF" -nt "$PORT_CONF" ]; then
>> > echo "Updating $PORT_CONF"
>> > sed "s/@port/$port/g" "$BASE_CONF" > "$PORT_CONF"
>> > fi
>> >
>> > PIDFILE=/var/run/mosquitto-$port.pid
>> >
>> > case "$1" in
>> > start)
>> > if init_is_upstart; then
>> > exit 1
>> > fi
>> > log_daemon_msg "Starting network daemon:" "mosquitto-$port"
>> > if start-stop-daemon --start --quiet --oknodo --background
>> > --make-pidfile --pidfile ${PIDFILE} --no-close --exec ${DAEMON} -- -c
>> > "$PORT_CONF" >> $LOG_PATH 2>&1 ; then
>> > log_end_msg 0
>> > else
>> > log_end_msg 1
>> > fi
>> > ;;
>> > stop)
>> > if init_is_upstart; then
>> > exit 0
>> > fi
>> > log_daemon_msg "Stopping network daemon:" "mosquitto-$port"
>> > if start-stop-daemon --stop --quiet --oknodo --pidfile
>> ${PIDFILE};
>> > then
>> > log_end_msg 0
>> > rm -f ${PIDFILE}
>> > else
>> > log_end_msg 1
>> > fi
>> > ;;
>> > ...
>> >
>> > Thanks very much.
>> >
>> >
>> >
>> >
>> >
>> > 2014-06-03 17:17 GMT+08:00 Roger Light <
>> <email address hidden>
>> >>:
>> >
>> >> Your question #249688 on mosquitto changed:
>> >> https://answers.launchpad.net/mosquitto/+question/249688
>> >>
>> >> Status: Open => Answered
>> >>
>> >> Roger Light proposed the following answer:
>> >> Can you provide any more details about your config please?
>> >>
>> >> On Tue, Jun 3, 2014 at 9:56 AM, jinlove
>> >> <email address hidden> wrote:
>> >> > New question #249688 on mosquitto:
>> >> > https://answers.launchpad.net/mosquitto/+question/249688
>> >> >
>> >> > I always came cross such a error in the log when i try to stop
>> mosquitto
>> >> with "/etc/init.d/mosquitto stop" :
>> >> >
>> >> > "mqtt3_context_cleanup: Assertion `context->listener->client_count >=
>> 0'
>> >> failed"
>> >> >
>> >> > I launch several mosquitto instances with different ports and
>> different
>> >> persistent files, and the error always(not every time) happens on every
>> >> instance after 3-4 days since last start, and the server just can't
>> start
>> >> again until i manually remove the persistence file.
>> >> >
>> >> > version : 1.2.3
>> >> > deploy on wheezy
>> >> >
>> >> > thanks.
>> >> >
>> >> > --
>> >> > You received this question notification because you are a member of
>> >> > Mosquitto PPA, which is an answer contact for mosquitto.
>> >>
>> >> --
>> >> If this answers your question, please go to the following page to let us
>> >> know that it is solved:
>> >>
>> >>
>> https://answers.launchpad.net/mosquitto/+question/249688/+confirm?answer_id=0
>> >>
>> >> If you still need help, you can reply to this email or go to the
>> >> following page to enter your feedback:
>> >> https://answers.launchpad.net/mosquitto/+question/249688
>> >>
>> >> You received this question notification because you asked the question.
>> >>
>> >
>> >
>> > --
>> > Lawrence
>> >
>> > You received this question notification because you are a member of
>> > Mosquitto PPA, which is an answer contact for mosquitto.
>>
>> --
>> If this answers your question, please go to the following page to let us
>> know that it is solved:
>>
>> https://answers.launchpad.net/mosquitto/+question/249688/+confirm?answer_id=2
>>
>> If you still need help, you can reply to this email or go to the
>> following page to enter your feedback:
>> https://answers.launchpad.net/mosquitto/+question/249688
>>
>> You received this question notification because you asked the question.
>>
>
>
> --
> Lawrence
>
> You received this question notification because you are a member of
> Mosquitto PPA, which is an answer contact for mosquitto.

Revision history for this message
jinlove (wonderfulseason) said : 		#6

Thanks for your hard work Roger. i will test it in the incoming version.

Can you help with this problem?

Provide an answer of your own, or ask jinlove for more information if necessary.

To post a message you must log in.