Finding in a region inside a captured image

Asked by Rafael Tomazela on 2017-12-29


I'm making a bot for a game (it's a match 3, like bejeweled/candy crush), and I'm trying to recognize the state of the game in a grid by looking at each of the grid cell and comparing with a set of predefined images.

So what I do is to loop for each cell in the grid (x, y, width and height) and check if each predefined image is in that region. The problem is that it's quite slow because I think it tries to capture the region each time I use .exists.

Since the game doesn't change while I don't make an action, I don't need to recapture the screen for each cell, I can capture only once and then search in regions of the image. Can I do that somehow?



Oh, and I'm using the python script inside the editor. But I can use something else like the java api if required.


Oh okay, actually I got it to work reasonably well, but I think there must be a better way. If there is not, maybe it's useful for someone with the same problem. What I did:

First I created a region, then I used `board = Screen().capture(region)`
Then I got the java BufferedImage with `javaimg = board.getImage()`
Then from the java image, I could get a subImage with subimg = javaimg.getSubimage(x, y, width, height)
Then, finally, I could recreated the ScreenImage with it ScreenImage(Rectangle(0, 0, width, height), subimg)
After that you can do what you want with the screen image. I used finder in my case.

It works reasonably fast!

Question information

English Edit question
Sikuli Edit question
No assignee Edit question
Last query:
Last reply:
RaiMan (raimund-hocke) said : #1

Ok, understood.

Congratulations for the way you found !!!

... but can it be, that you did not set the timeout with the exists to 0 ?
If not: in case of not found, it tries for 3 seconds before giving up.

This would have been my solution

for line in range(lines): # number of grid rows
    for cols in range cols: # number of cols
        # calculate the grid region on screen for this gridelement
        for image in gridimages: # gridimages contains the images to check
            if gridRegion.exists(image, 0):
                # do what needed

with version 1.1.2 there are findBest and findAny, that might help in your case:

Can you help with this problem?

Provide an answer of your own, or ask Rafael Tomazela for more information if necessary.

To post a message you must log in.