Capturing the Pattern found as part of the Match object returned

Asked by skeets

Is there any thought about adding the Pattern to the Match object returned from a find, exists, etc? We have a high rate of screen image changes and to try to capture the screen AFTER the match has been found is not working consistently. We would like to capture exactly the Pattern Sikuli found and in some cases search that pattern for a sub-image.

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
skeets
Solved:
Last query:
Last reply:
Revision history for this message
Eugene S (shragovich) said :
#1

Well, you can't get a match if the actual Pattern wasn't found on the screen. The match object will only be created AFTER the pattern was matched on the screen/region.
If you want just physically see what was visible on the screen in a specific point in time you can just grab a screenshot and save it in file.

You can do it like that:

import shutil

img = capture(SCREEN) # img is an image file .png in temp folder now
shutil.move(img, <tagetPath>)

Eugene

Revision history for this message
RaiMan (raimund-hocke) said :
#2

@ skeets
good point.

I will make this a request bug:
a match should remember the image, that was actually matched and the similarity minimum level used for the search.

This will either be available already in 1.1.0 final or early in 1.2.0-Beta.

Revision history for this message
skeets (skeets-malloy) said :
#3

RaiMan, thanks for the quick turnaround! :)

Eugene, yeah we already do that. If it matches, we screen capture the region Sikuli thinks it matches. BUT by the time Sikuli says it matches and the time we have Sikuli take a screen capture of that region, the screen has changed. We deal with quickly changing images. If Sikuli doesn't find a match we screen capture there too.

We write out a test results HTML file that shows our expected image versus the actual Sikuli found image, with pass/fail and min similarity Sikuli used for considering it a Match.

Also, as I mentioned above, we are thinking about doing "sub" searches on the Patterns returned.

Thanks again all.

Revision history for this message
skeets (skeets-malloy) said :
#4

I'm back! :D Looking at the new documentation, I'm wondering if this is updated as expected? I see that Match does have some new methods. I was thinking that the Match object would retain the "Sikuli found" pattern that matched the expected Pattern that was passed into Region.exists(). I think Match is retaining the expected image passed in?

I do see the new methods in Region that return the last screen image (and file). I did test out getting the Match last screen image and still have issues writing out to HTML my expected image (that I passed into Region.exists()) and the actual image Sikuli found. I still see that the expected image and last screen image do not show the same thing even though the match score was over 0.99. It is WAY better than before where almost all of the time we could not capture the screen image after the exists() call as the image was already gone but not 100% as I thought would be.

I could be jumping the gun but wanted to ask in case I might be missing something. THANKS!!

Revision history for this message
RaiMan (raimund-hocke) said :
#5

-1: yes, a Match object now carries the information about the image used for finding (an object of class Image).
Match.getImage() and Match.getImageFilename()

- 2. screen capture image used to find the given image
… currently always is the one taken at the last capture in any region belonging to that screen.

So this is only the "same" directly after a successful find.

-- same:

if exists(some_image):
    print getLastMatch().getImage()
    print getLastScreenImage() # this should contain some_image

-- not same:

if exists(some_image):
    success = getLastMatch().getImage()
    if exists(some_image1):
         # still the same, since screen should not have changed
         # different if screen meanwhile changed for some reason

In version 2 I will add some "visual logging".

Revision history for this message
skeets (skeets-malloy) said :
#6

Maybe we are doing something wrong in how we are using it? Here is our example:

expectedImgPath = c:\path_to\expected_image\to_find_on\screen.png
minSim = 0.99

# create pattern to search for
expectedPattern = Pattern(expectedImgPath).similar(minSim)

# see if a Match exists, of the expectedPattern, in the window
imgMatch = theWindowToSearch.exists(expectedPattern)

if (imgMatch is not None):
    # get the screen image path Sikuli used to find the Match of the expectedPattern
    resultImgPath = imgMatch.getLastScreenImageFile() # not always the same as expected image

We then go on to add results to an HTML file showing the expectedImgPath in one column, resultImgPath in another and then the match score in the last column. This is where we can sometimes see that the expected image is not seen in the result screen image even though the match score is showing over 0.99. Again for the MOST part, say 95% of the time, it appears to be working, which is WAY better than before.

Our expected image that is found is only on screen for 0.5 seconds and then disappears if that matters? We are basically testing that an icon is blinking.

Thanks again.

Revision history for this message
RaiMan (raimund-hocke) said :
#7

Just checked.

My test and hence a use case can be found in
https://github.com/RaiMan/SikuliX-2014/blob/master/TestRunMaven/src/main/java/com/sikulix/testrun/TestRun.java

The images are contained in the project.

IMPORTANT: do not use the highlight() function. This currently changes the lastScreenImage. I will fix this.

Revision history for this message
skeets (skeets-malloy) said :
#8

OK we have test output that can show you a bit more about what we are seeing with this latest 1/16/15 release. It seems that the first few getLastScreenImages are OK but after that it appears that some getLastScreenImages are "lingering" around so a screen image from a PREVIOUS test is printed out to our HTML test result file instead of the CURRENT getLastScreenImage. We don't reuse any match object or anything. It appears to work for one or two of our test steps and then the next step it doesn't work, then it works again, etc.

All our code does is take the Match Image and call getLastScreenImageFile() and then we put that file into our HTML test result file. We do this after every test step. The code example is what I put up in #6 above. That is all we really are doing.

Is there a way to send you/attach the result HTML file to show you what I mean? Thanks much.

Revision history for this message
RaiMan (raimund-hocke) said :
#9

you can send what you want to my mail at https://launchpad.net/~raimund-hocke (top left corner).
preferably as zip file

some code sample would help too, so I can see how you use it.