Optimization on targeting moving button/picture

Asked by leocd

Hi, first of all, great job!

Do you have any tips on targeting any moving button/picture?

I usually use :

     if reg.exists(a,0)
        reg.click(a,0) # slow and sometimes hit and miss on target

than using :

    if reg.exists(a, x):
        click(reg.getLastMatch()) # always miss, I think because the button/picture position already changed.

But it is still kinda slow.

Thanks.

Question information

Language:
English Edit question
Status:
Answered
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
leocd (sleeplesscat91) said :
#1

How about this for an idea..

Every time the program find the moving picture using exists(), it will make a small region around to be search again then click(). How's that?
How to implement that in code?

I tried this but didnt work :

     if reg.exists(a,0)
         reg_found = (getLastMatch(),50,50); # creating region from last known picture position
         reg_found.click(a,0)

Revision history for this message
RaiMan (raimund-hocke) said :
#2

This is valid for version 1.1.1

The internal approach, to find an image is as follows (pseudo code)

loop:
    take a screenshot of the search area (1)
    search for the target image in the screenshot (2)
    if found leave the loop
    else if waittime exceeded break the loop (always true for waittime 0)
    else continue the loop

(1) takes about some 10 milliseconds depending on size of search area
(2) takes some 10 milliceconds up to some 100 milliseconds (slower with larger search areas, faster if search area size is similar to target image size). A search in large screens may take more than 500 milliseconds on slow systems.

So the strategy for clicking moving objects is as follows:

# initial wait for the target img:
if maxReg.exists(img, maxWaiTime):
# if maxReg is the screen: if exists(img, maxWaiTime):

    # get the area, that the possibly moved target should be found:
    possibleReg = maxReg.getLastMatch().grow(50)
    # 50 pixels are added around the last match, must be adjusted depending on timing
    # if maxReg is the screen: possibleReg = getLastMatch().grow(50)

    if possibleReg.exists(img, 0): # only one very fast search
        possibleReg.click() # this will click the last match in possibleReg
    else:
        print "not found after move, increase possible region"

else:
    print "initial wait for appearence exceeded, increase maxWaitTime"

In all cases it is vital to have good screenshots of the target image, that result in find scores beyond 0.9 - 0.95
see: http://sikulix-2014.readthedocs.io/en/latest/basicinfo.html#sikulix-how-does-it-find-images-on-the-screen

Can you help with this problem?

Provide an answer of your own, or ask leocd for more information if necessary.

To post a message you must log in.