[HowTo] search/wait for multiple images in parallel

Created by RaiMan on

*** This faq is motivated by the following question, that was addressed to me personally:

I have a function in the Library.py outside the main that searches for 16 images - and if any of them is found it turns a flag to True and returns the Flag.

# -----Library.py
from sikuli.Sikuli import *
image1= (....png)
image16 = (....png)
def CK():
flag = False
if exists(image1, 0):
   flag = True
if exists(image16, 0):
   flag = True
return flag
#------ Main
# imports the Library
button1 = (.....png)
button2 = (.....png)
if CK():
#execute1 - click(button1)
#execute2 - click(button2)

Issue1: the function returns very slow and the execute1 or execute2 are done after aprox 20 secs - can I change the code to make the searches faster (5 secs max)?

Issue2: how can I make my function to return after finding the 1st image? and not continue searching through all 16?


*** this was my answer:

before reading further, make sure you are familiar with the facts in faq 1607
[HowTo] make scripts fast and robust against intermittent FindFailed exceptions

**** a more general solution for your def CK() and issue2 is solved

based on my answer to https://answers.launchpad.net/sikuli/+question/167084
the sophisticated version for your approach.

Wether the def CK() is in a library or not does not really matter.

def CK(imgs, t = None, first = False, reg = None):
   if not t: t = 2 * len(imgs)
   if not reg: reg = SCREEN
   res = [False for img in imgs]
   start = time.time()
   while time.time()-start < t:
       for i in range(len(imgs)):
           if reg.exists(imgs[i], 0):
               res[i] = True
               if first: return res
       if True in res: return res
   return None # nothing found within time

usages for searching whole screen:
--- search all, max time = 2 * (number of images) seconds
retVal = CK(listOfImages)
--- stop at first find, max time = (number of images) seconds
retVal = CK(listOfImages, first = True)
--- search all, max time given
retVal = CK(listOfImages, t = 20)
--- stop at first find, max time given
retVal = CK(listOfImages, t = 20, first = True)

usage for a region on the screen (faster):
in all the above cases just specify additionally the reg parameter as:
retVal = CK(listOfImages, reg = some_region)

*** the main workflow (supposing it is on Windows)

win = App("some applications window title").window() # restrict to app window
imgs = [img1, img2, img3] # images, patterns or texts

retVal = CK(imgs, reg = win)
if not retVal:
    print "waiting time exceeded"
    print "found something:", retVal

# now you may inspect retVal
if retVal[0]:
   # special action for image 1

**** your issue1
Searching on the whole screen costs about 1 second in average on a screen with actual resolutions (average 1200-1400 x 800-1000). The only chance to boost search performance is to restrict the search area. Mostly already restricting to the app window (not being fullscreen) brings about 30%. If it is possible to restrict the search area to a region of about 300 x 300, the average search time might decrease to an average below 0.5 seconds.

So in general when designing a workflow like yours it is a good approach to calculate with an average of 1 second per search action. The experience on my big machine (MacPro 1st gen, 2,66 GHz, 11 GB), is that during the standard waiting time of 3 seconds 2 - 4 searches are performed on the whole screen (1900x1200) depending on the size of the images.

So in your case with 16 images and a restricted region, the possible optimum when searching all I guess will be about 10 seconds. So if you arrange the images according to their probability of appearance and use first = true, you might have a chance to reduce the average to 5 seconds.

A real boost in performance can only be reached, when delegating the searches to subprocesses. You might try this with the observe() feature (possible, but I do not recommend because of the rather complex coding). I would prefer to simply use the threading feature of Python (http://www.jython.org/docs/library/thread.html).