Algorithm: find from within a list of images

Asked by alex on 2018-07-17

What I would like to do:
-I have a list of images (in the order of 30-50) that may show up on the screen
-The screen may show a certain sub-group of these images at the same time, typically 8-12 of them
-I need to click on any one of the images on the screen that is contained in my list
-The order does not really matter. Neither which one of the images on the screen is clicked nor which position it has within the list

Any chance this can be done? I am also a bit worried about performance since my list of possible matches is rather long. That could result in quite a few find operations.

Question information

Language:
English Edit question
Status:
Solved
For:
Sikuli Edit question
Assignee:
No assignee Edit question
Solved by:
RaiMan
Solved:
2018-07-18
Last query:
2018-07-18
Last reply:
2018-07-18
Alex (carroll-alex-d) said : #1

This is pretty straight forward, if I understood your question completely:

1. Generate a list of your images:
image_search_list = ['img1.png', img2.png', img3.png'...]

2. Cycle through this list in a for loop:

for image in image_search_list:
    if exists(image, 0.2):
        click(image)
        break

This will cycle through every image in your list. If it is detected, it is clicked. The loop is broken and stops searching after one image is found. You can wrap that in it's own loop with the necessary waits and clicks to get back to this search. You will likely need to play with the search time. I defaulted to 0.2.

You can, and likely should make refinements to the process with regions and patterns. Regions can limit your search area which can make your searches faster. You made need to refine the image as a pattern to set your image match percentage (default is a 70% match). Lower the match percentage can speed up the process but may introduce false positives. Again, you will need to experiment.

I can answer more questions if you further articulate the question.

alex (privatesnowball) said : #2

Thanks, that definitely does what I need.
This seems like a "brute force" approach to the problem. I will have to think about it for a while if there is a more elegant (aka faster) way to do this.

Best RaiMan (raimund-hocke) said : #3

may be this is a bit more straight forward:
http://sikulix-2014.readthedocs.io/en/latest/region.html#find-more-than-one-image-in-a-region-at-the-same-time

This is more compact and faster than a series of exists(image, 0), since each find op is run in parallel in threads.
(faster for more than 3 - 5 images depending on the size of the search region)

alex (privatesnowball) said : #4

Thanks RaiMan, that solved my question.

Alex (carroll-alex-d) said : #5

Thanks RaiMan, I need to take a closer look at the new toys in 1.1.2. That was much more elegant than my solution.