Sikulix Region not working

Asked by Jasper

I've been trying to make a script that looks for specific images, clicks on them, and then clicks on the next button after that.
Currently, it does that - but it's incredibly slow, it can take up to two minutes just to click on 4-6 images/buttons. Overall there are about 10 images it's looking for all at once as they randomly pop up.

I've tried adding a region to my code to help speed it up but I couldn't figure it out as I'm extremely new to even the most basic of script writing. Below is the code I'm currently using:
while True:
    Settings.ObserveScanRate = 18
    Settings.WaitScanRate = 15
    Settings.MoveMouseDelay = 0
    Settings.MouseDelay = 0

    CurrentBarRegion = ( Region(689,294,1059,685))

    if exists (Pattern("1452165008359.png").similar(0.76), 0): hover (getLastMatch())
    if exists ("1452165060367.png", 0): click(getLastMatch())

There's about eight more if exists with pictures in it but that's the code.
Some thought as to why Sikulix is slow: all images are very similar with light blue colors, seems Sikulix doesn't like light blue that much. Maybe the method I'm using is bad and slowing down the process as well. Could try using text search possibly, all buttons have big text in them.

 Pictures are all high quality, my monitor is 24 inches, 1920x1080 resolution, lots of ram good CPU etc so it can't be due to that. If anyone can help me speed up Sikulix by showing me how to put in region properly or another method I'd greatly appreciate it. Thanks!

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said :
#1

Supposing you are using version 1.1.0

To get the speed-up of using Regions:

if CurrentBarRegion.exists(Pattern("1452165008359.png").similar(0.76), 0): CurrentBarRegion.hover()

... which searches in the CurrentBarRegion and if found hovers over the last match in CurrentBarRegion

... and you should just work with the standard Settings especially as newbie:

so just purge these:
    Settings.ObserveScanRate = 18 # not relevant for your script (no observe used)
    Settings.WaitScanRate = 15 # does not have an effect with exists(..., 0)
    Settings.MoveMouseDelay = 0 # saves 0.5 seconds per mouse move (not really relevant)
    Settings.MouseDelay = 0 # not relevant in your case and seldom necessary

Revision history for this message
RaiMan (raimund-hocke) said :
#2

... and the same for this:

 if exists ("1452165060367.png", 0): click(getLastMatch())

... better

 if CurrentBarRegion.exists("1452165060367.png", 0): CurrentBarRegion.click()

Revision history for this message
Jasper (masterjasper19) said :
#3

Thanks for the code it worked - however, now after about fifteen minutes or so it crashes and I get Java(TM) Platform SE binary has stopped working, aside from that it works.. any idea why it's doing that? here is the error message I got:

OpenCV Error: Insufficient memory (Failed to allocate 6080476 bytes) in OutOfMemoryError, file c:/MinGW/msys/1.0/stuff/opencv/modules/core/src/alloc.cpp, line 52
terminate called after throwing an instance of 'cv::Exception'
  what(): c:/MinGW/msys/1.0/stuff/opencv/modules/core/src/alloc.cpp:52: error: (-4) Failed to allocate 6080476 bytes in function OutOfMemoryError

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

Don't exactly know why it's doing that but I have 16 gigs of ram, and 4gb of vram on my GPU if that helps.

Revision history for this message
Jasper (masterjasper19) said :
#4

Found the issue - I had multiple heavy scripts running at the same time apparently, I thought sikulix only ran the script you had selected not all of the tabs too. Newbie skills at work :)
Anyways, everything is working like a charm now. Thanks for the help RaiMan!

Revision history for this message
Jasper (masterjasper19) said :
#5

Thanks RaiMan, that solved my question.

Revision history for this message
Jasper (masterjasper19) said :
#6

Hmm, apparently I jumped the gun with saying my issue was solved. Sadly it's not - I'm still getting out of memory errors and having it crash on me. After some research people are saying its CurrentBarRegion =( Region(684,285,1068,710)) about it not releasing heap memory after it finishes a task, my script is on an infinite loop by the way.

Maybe I need something to tell it to dump the heap memory or something? Before I was using CurrentBarRegion =( Region(684,285,1068,710)) on my older script I had no issues at all and it ran for over 8 hours straight before I stopped it. Definitely seems like its the region that's causing the memory crashes.

Sorry for the spam comments, I got ahead of myself thinking it was fixed when it was not. Anyways this has me stumped so I hope you have a solution! Thanks again for your help.

Revision history for this message
RaiMan (raimund-hocke) said :
#7

Please send the zipped .sikuli folder to my mail rmhdevelop--at--me--dot--com.

I have to check.

On what system are you running? What Java version?

Still supposing it is version 1.1.0

Revision history for this message
Jasper (masterjasper19) said :
#8

Okay, sent! I was on version 1.1.0 yes - I tried the nightly version 1.1.1 and it is still doing it, seems to take much longer on the nightly version, though. This time, it took about 3 hours before I had a no memory error.

System specs: Windows 10 Home build 10586 - CPU is i7-4790k - motherboard and GPU: Evga z97 classified - GTX 670 4gb ftw

Java version: 1.8.0_66-b18 I believe it is the newest one as I updated it last night. Crashed on older java and this one as well.

Revision history for this message
RaiMan (raimund-hocke) said :
#9

testing with a similar environment, I cannot see, that the memory consumption constantly increases over time.

Nevertheless, you should take
CurrentBarRegion =( Region(684,285,1068,710))

out of the loop (hence before the while), since it is a constant and no need to create the same new Region object with every loop turn.

another question:
the crash is reported in
c:/MinGW/msys/1.0/stuff/opencv/modules/core/src/alloc.cpp

Do you have this on your system?

Revision history for this message
Jasper (masterjasper19) said :
#10

Just checked - I can't find it at all for some reason. Two Microsoft folders came up when I just looked for MinGW though, but it had nothing in them but SO files in a bunch of different folders and stuff.

Did you do the test with current bar region inside the loop or did you take it out of the loop before the test?
I'm not sure why there is no log at all, it's crashed on me a few times now. There should be something.

I just edited the script with your suggestion, maybe that is what was causing it to crash? If it crashes again, I'll make sure to triple read everything that pops up in cmd and see if there's another dump location for any crash reports.

Revision history for this message
RaiMan (raimund-hocke) said :
#11

--- MinGW folder
I only wanted to make sure.
The information seems to be the debug information, that is preserved in the pre-built stuff from the time when I did the build on my system. So just forget it.

--- short on memory exception
the problem situation as such is clear and does not need any further debugging: at the time of crash, OpenCV tries to allocate memory for a screen image that was captured before, to search something in it. This is the normal procedure during SikuliX find operation.

The question is: Why does memory get short in your situation? Usually this is caused by some memory leak (Java objects still referenced somehow, so their memory cannot be garbage collected).
This might be information, that does not say anything to you ;-)

So just make your test with the statement outside the loop and report back.

Hope I will find something meanwhile or in the next days.

Revision history for this message
Jasper (masterjasper19) said :
#12

So it's been about 8 hours now since I started my script and no crash at all. I did refine the script a bit by reducing the current bar region by about half of what it was and resized most of the pictures to be smaller as some had the same first couple words in them which made a good chunk of the pictures redundant as only a couple are needed to register the rest.

As of right now it seems fixed, but I did reduce the size of my script by a lot so it is possible it just now takes longer to reach the memory cap? Not sure if the size of the region/script has anything to do with it or not. If not, then it's definitely fixed.
If that can affect the time it takes to reach the memory "cap" then it's possible I just haven't reached it yet.

 , I'm going to let it continue running until it reaches a full 24-hour cycle if it reaches that I'll assume it's fixed.
Or if you can confirm the above, about the sizes that it does or does not matter would also confirm if its fixed or not.

So far It looks like the culprit was having the region inside the loop, but I'm very new to this so I just want to be positive that is the case by being thorough as possible.

Revision history for this message
RaiMan (raimund-hocke) said :
#13

Thanks again for your valuable feedback, highly appreciated.

... and what you have done so far with your changes moves you some steps forward towards being a SikuliX expert ;-)

I will create my own test, that can run over a longer time and reports constantly about the different memory consumptions.
And I will check the code, wether there might be some possible memory leak.

To track this case, I will convert your question to a bug.

Revision history for this message
RaiMan (raimund-hocke) said :
#14

made it a bug for further treatment

Revision history for this message
Jasper (masterjasper19) said :
#15

No problem! Honestly, thank you for showing me how to use the Region with my code, helped out a lot! As a new coder, little things like that can be invaluable haha.

The script still is running very smooth, no issues at all. It hasn't been 24 hours yet but I am positive it's fixed. the issue definitely was having the loop create the same Region object with every loop turn, That was causing the no memory crashes.

After taking it out of the loop as you suggested the issue stopped so thanks for that too, and actually making SikuliX and maintaining it as well. :)

Maybe one day I will be an expert at SikuliX I've always been fascinated with automating things which lead directly to coding. For me learning to code is a lot fun, even though I suck at it at the moment. But for now, my newbie skills will have to suffice.

 well, I guess back to learning more about SikuliX and coding! Thanks again for all of your help, it really was invaluable!