mysqld process cpu usage

Hi Pascal

I ran in to this problem yesterday as it happens. On one of your clients, a backlog of statistics has built up - possibly due to an error (we're still looking at that) - which is causing the client to repeatedly log stats back to the webservice.

On each client, look in C:\Documents and Settings\user\Application Data\Xibo\Xibo Digital Signage\2.0.0.0

One of them will probably have a whole load of .ready files. Stop that client running and delete all the files in that directory (including any .xml files).

It would be interesting to get the full output from your SQL logging and the errors you mention in the log. Please could you save those out as text files, compress them and then email to <email address hidden>?

Many thanks

Alex

-------- Message original --------

> Your question #76281 on Xibo changed:
> https://answers.launchpad.net/xibo/+question/76281
>
> Status: Open => Answered
>
> Alex Harrington proposed the following answer:
> Hi Pascal
>
> I ran in to this problem yesterday as it happens. On one of your
> clients, a backlog of statistics has built up - possibly due to an error
> (we're still looking at that) - which is causing the client to
> repeatedly log stats back to the webservice.
>
> On each client, look in C:\Documents and Settings\user\Application
> Data\Xibo\Xibo Digital Signage\2.0.0.0
>
> One of them will probably have a whole load of .ready files. Stop that
> client running and delete all the files in that directory (including any
> .xml files).
done

>
> It would be interesting to get the full output from your SQL logging and
> the errors you mention in the log. Please could you save those out as
> text files, compress them and then email to <email address hidden>?
>
> Many thanks
>
> Alex
>
you want the /var/log/mysql/mysql.log files and the log table ?

thanks

--

---------------------------------------------------------------
Pascal Legrand
*IUT de Chartres* - _Service Informatique_
---------------------------------------------------------------
1, place Roger Joly
28000 Chartres
Tel : 02 37 91 83 36 - Fax: 02 37 91 83 01

-------- Message original --------

> Your question #76281 on Xibo changed:
> https://answers.launchpad.net/xibo/+question/76281
>
> Status: Open => Answered
>
> Alex Harrington proposed the following answer:
> Hi Pascal
>
> I ran in to this problem yesterday as it happens. On one of your
> clients, a backlog of statistics has built up - possibly due to an error
> (we're still looking at that) - which is causing the client to
> repeatedly log stats back to the webservice.
>
> On each client, look in C:\Documents and Settings\user\Application
> Data\Xibo\Xibo Digital Signage\2.0.0.0
>
> One of them will probably have a whole load of .ready files. Stop that
> client running and delete all the files in that directory (including any
> .xml files).
done

>
> It would be interesting to get the full output from your SQL logging and
> the errors you mention in the log. Please could you save those out as
> text files, compress them and then email to <email address hidden>?
>
> Many thanks
>
> Alex
>
you want the /var/log/mysql/mysql.log files and the log table ?

thanks
ps the client is not running and mysqld use a lot of cpu

--

---------------------------------------------------------------
Pascal Legrand
*IUT de Chartres* - _Service Informatique_
---------------------------------------------------------------
1, place Roger Joly
28000 Chartres
Tel : 02 37 91 83 36 - Fax: 02 37 91 83 01

Yes - both please.

If the client isn't running now, I expect there's still an error causing this loop. Please stop Apache (by killing it if required) so that the MySQL loading goes back to nothing. You can then sort out the client causing the problem.

Cheers

Alex

-------- Message original --------

> Your question #76281 on Xibo changed:
> https://answers.launchpad.net/xibo/+question/76281
>
> Status: Open => Answered
>
> Alex Harrington proposed the following answer:
> Yes - both please.
>
> If the client isn't running now, I expect there's still an error causing
> this loop. Please stop Apache (by killing it if required) so that the
> MySQL loading goes back to nothing. You can then sort out the client
> causing the problem.
>
> Cheers
>
> Alex
>
here is the log table and a piece of my mysql.log

however, the problem seems disapered since i restart apache ??

i've got only one client running

thanks

--

---------------------------------------------------------------
Pascal Legrand
*IUT de Chartres* - _Service Informatique_
---------------------------------------------------------------
1, place Roger Joly
28000 Chartres
Tel : 02 37 91 83 36 - Fax: 02 37 91 83 01

Hi Pascal

Launchpad doesn't support email attachments - please could you send your
email to <email address hidden>

Cheers

Alex

hello,
in fact the problem is still present : the mysqld process still use 100% of cpu each time xibo start or if i made some changes

thanks

Hi Pascal

From my previous reply:

> On each client, look in C:\Documents and Settings\user\Application Data\Xibo\Xibo Digital Signage\2.0.0.0
>
>One of them will probably have a whole load of .ready files. Stop that client running and delete all the files in that directory (including any .xml files).

Please can you check for those again and let me know if you find any (and if so how many).

Cheers

Alex

-------- Message original --------

> Your question #76281 on Xibo changed:
> https://answers.launchpad.net/xibo/+question/76281
>
> Status: Open => Answered
>
> Alex Harrington proposed the following answer:
> Hi Pascal
>
>>From my previous reply:
>
>> On each client, look in C:\Documents and Settings\user\Application Data\Xibo\Xibo Digital Signage\2.0.0.0
>>
>>One of them will probably have a whole load of .ready files. Stop that client running and delete all the files in that directory (including any .xml files).
>
> Please can you check for those again and let me know if you find any
> (and if so how many).
>
> Cheers
>
> Alex
>
there is only one file : 128919385092343750.ready

thanks

--

---------------------------------------------------------------
Pascal Legrand
*IUT de Chartres* - _Service Informatique_
---------------------------------------------------------------
1, place Roger Joly
28000 Chartres
Tel : 02 37 91 83 36 - Fax: 02 37 91 83 01

OK - so can you rename it to something other than .ready (eg .ready.old), delete all the other files, restart Apache and then start the client back up please.

Alex

-------- Message original --------

> Your question #76281 on Xibo changed:
> https://answers.launchpad.net/xibo/+question/76281
>
> Status: Open => Answered
>
> Alex Harrington proposed the following answer:
> OK - so can you rename it to something other than .ready (eg
> .ready.old), delete all the other files, restart Apache and then start
> the client back up please.
>
> Alex
>
i already done that last time, it worked and after the problem come back
any way i do it

i stop the client
rename 128919385092343750.ready => 128919385092343750.ready.old
rename 128919436651093750.ready => 128919436651093750.ready.old
delete log.xml, schedule.xml
restart apache
restart the client

thanks

--

---------------------------------------------------------------
Pascal Legrand
*IUT de Chartres* - _Service Informatique_
---------------------------------------------------------------
1, place Roger Joly
28000 Chartres
Tel : 02 37 91 83 36 - Fax: 02 37 91 83 01

Hi Pascal

I'm not clear. Doing that has solved it again for now, or is it still doing it?

We haven't made any change yet to prevent that from happening - as we're still trying to fully understand what causes it (and in such a small number of cases - as far as I'm aware the only people to see this are you and me - and I've not had any recurrance).

Could you email over those renamed .ready files so we can see what's in them.

Cheers

Alex

-------- Message original --------

> Your question #76281 on Xibo changed:
> https://answers.launchpad.net/xibo/+question/76281
>
> Status: Open => Answered
>
> Alex Harrington proposed the following answer:
> Hi Pascal
>
> I'm not clear. Doing that has solved it again for now, or is it still
> doing it?
no problem, you are clear, it's only my poor english wich is the problem
it solve the problem only for a moment, some time after the problem come back
>
> We haven't made any change yet to prevent that from happening - as we're
> still trying to fully understand what causes it (and in such a small
> number of cases - as far as I'm aware the only people to see this are
> you and me - and I've not had any recurrance).
>
> Could you email over those renamed .ready files so we can see what's in
> them.

>
> Cheers
>
> Alex
thanks for your help
>

--

---------------------------------------------------------------
Pascal Legrand
*IUT de Chartres* - _Service Informatique_
---------------------------------------------------------------
1, place Roger Joly
28000 Chartres
Tel : 02 37 91 83 36 - Fax: 02 37 91 83 01

OK - thanks.

If you can email over those files to <email address hidden> we can put them in to a client here and see if we can reproduce it.

Cheers

Alex

Hi Pascal,

I didn't realise the .ready files were so large... .ready files should be no bigger than 6KB in size. It appears that there may be some reason the Xibo Client cannot create a ready file when it ought to.

Could you try disabling stats on that troublesome client? And then see if the problem reoccurs...

Do you have some media on any of your layouts that has a very low duration?

I will look into the code to see why the file is getting so large.

Cheers,
Dan

Hi Pascal,

I have found the problem and am now sure that turning stats off should resolve it for you.

If you still experience the problem with stats off please let me know...

Cheers,
Dan

We just ran into this problem today while testing. The client install was on a windows 7 box, not sure if that makes a difference or not. Took my server down twice today though, haven't seen the problem since disabling stats on it.

Is there a fix for the problem or just disable stats for the time being? What kind of reporting is there currently on the stats that are gathered?

Hi Andy,

Interesting to know that it runs ok on a windows 7 box!

We seem to have isolated the problem to the stats message and are in discussion about how best to fix it. I think the solution will turn out to be a slight re-think in the way we collect and send the stats. For the time being the best thing you can do is disable the stats.

Currently the server interface does not expose stats in any way, nor does it provide a method of archiving the stats.

The long and the short of it is that currently you cannot see the stats in the released version and they do sometimes cause problems. The only reason we will keep them in 1.0.3 (adjusted so they are fixed) is that we know of some people that have built interfaces ontop of them.

Hope this helps...

Hi Andy

We're pretty sure we understand the problem now and there will be a fix in 1.0.3. We don't do anything with the stats collected at the moment - except log them.

At some point a reporting interface will be written, and assuming you've had stats on then there will be historical data in there to report on.

Cheers

Alex

--- original message ---
From: "Andy" <email address hidden>
Subject: Re: [Question #76281]: mysqld process cpu usage
Date: 20th July 2009
Time: 9:50:07 pm

Question #76281 on Xibo changed:
https://answers.launchpad.net/xibo/+question/76281

Andy requested for more information:
We just ran into this problem today while testing. The client install
was on a windows 7 box, not sure if that makes a difference or not.
Took my server down twice today though, haven't seen the problem since
disabling stats on it.

Is there a fix for the problem or just disable stats for the time being?
What kind of reporting is there currently on the stats that are
gathered?

--
You received this question notification because you are a member of Xibo
Developers, which is an answer contact for Xibo.

Actually just had to come back in to the office, server was down again. I didn't delete the .ready file from the user folder though earlier, apparently the client will still try to process that even when stats are disabled?

Anyway, I've got the client up and running again but wanted to verify my oversight one way or the other. Stat table is happily unchanging right now so I imagine that was the problem.

Thanks for the info guys!

-andy

Andy,

Yes it will still process that file even if stats are disabled. Mainly because that file is also used for error logging - so last night the error logging would have triggered the send of that file.

Part of our proposed change is to separate the stats from the other logging and improve the situation where the client cannot send the log properly (causing your problem).

If it looks like 1.0.3 is going to take a while after this fix is available I will attach it to this thread so you can patch.

Cheers,
Dan