region.find(image) question/issue

Asked by Gary Paschke

SikulixIDE 1.1.1
Windows 10 64bit

I'm trying to compare the exact region currently displayed (via the sikulix script) against a saved image, which has the exact region size.

(I actually created the region within sikulix and saved to folder: D:\\Tc_Auto_Tests\\Images\\3D_Reciprocating_Saw_Gold.png").

Please let me know how to upload images for reference.

Script:

image="D:\\Tc_Auto_Tests\\Images\\3D_Reciprocating_Saw_Gold.png"
region=Region(351,248,1136,610) # this is the exact size of the saved (Gold) image size I want to find (or not find)
if region.find(image):
    print "Screen matches 3D_Reciprocating_Saw_Gold, test passed"
    region.highlight(2)
else:
    print "Screen failed match of 3D_Reciprocating_Saw_Gold, test FAILED"

Message:

[log] CLICK on L(138,78)@S(0)[0,0 1920x1080] (521 msec)
[log] TYPE "D:\Tc_Auto_Tests\TcVis_Data\3D\Reciprocating_Saw.jt#ENTER."
[log] RIGHT CLICK on L(900,500)@S(0)[0,0 1920x1080] (521 msec)
[log] CLICK on L(1017,592)@S(0)[0,0 1920x1080] (1521 msec)
[log] CLICK on L(85,35)@S(0)[0,0 1920x1080] (521 msec)
[log] CLICK on L(300,86)@S(0)[0,0 1920x1080] (521 msec)
[log] CLICK on L(544,61)@S(0)[0,0 1920x1080] (521 msec)
[log] CLICK on L(400,400)@S(0)[0,0 1920x1080] (522 msec)
[log] TYPE "Test a 3D Markup Text#ENTER."
[log] CLICK on L(515,516)@S(0)[0,0 1920x1080] (521 msec)
[log] CLICK on L(300,86)@S(0)[0,0 1920x1080] (520 msec)
Screen matches 3D_Reciprocating_Saw_Gold, test passed -- Don't expect this as find should not match current resion with saved image
[log] highlight R[351,248 1136x610]@S(0) for 2.0 secs

Thanks for all your help,
Gary Paschke

Siemens Product Lifecycle Management Software Inc.
Lifecycle Coll
5939 Rice Creek Parkway
Shoreview, MN 55126 United States
Tel. :+1 (651) 855-6162
Fax :+1 (651) 855-6285
<email address hidden>

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
Gary Paschke
Solved:
Last query:
Last reply:
Revision history for this message
Gary Paschke (garypaschke) said :
#1

I meant to add that the 3D_Reciprocating_Saw_Gold.png does NOT match the current region (351,248,1136,610), as I changed the image appearance (not size/region). So, I'd expect the find to fail and print "Screen failed match of 3D_Reciprocating_Saw_Gold, test FAILED"

-Gary Paschke

Revision history for this message
RaiMan (raimund-hocke) said :
#2

--1:
instead of
 region.find(image)

you should use
 region.exists(image)

in if/elif/while ... since it returns True or False instead of throwing a FindFailed exception

--2:
 region.find(image)
is successful if the match score is >0.7 (standard behavior).
Be sure you have understood the match score concept, that is implemented with class Pattern.
In your case the image might have changed a little, but still matches with a score > 0.7

Revision history for this message
Gary Paschke (garypaschke) said :
#3

I changed script to below and still do not get my expected (desired) results

image="D:\\Tc_Auto_Tests\\Images\\3D_Reciprocating_Saw_Gold.png"
r = Region(200,200,1700,825)
pimage = Pattern(image).exact()

if r.exists(pimage, 0):
   print "Screen matches 3D_Reciprocating_Saw_Gold, test passed"
   r.highlight(2)
else:
   print "Screen failed match of 3D_Reciprocating_Saw_Gold, test FAILED"

What I need to do, is check the current screen against a saved (3D_Reciprocating_Saw_Gold.png) image. The Region will never change, the image could change, which is what I need to check. The region is the entire working area that the image will reside inside of.

Revision history for this message
RaiMan (raimund-hocke) said :
#4

- your region is an area on the screen top left at (200, 200) with a size of (1700 x 825)

- your image is a png image file at the given path

- with Pattern(image).exact() you say, that the image should be seen in the region exactly as it is at time of region.exists()

- at time of region.exists() SikuliX makes a screenshot of the region area and checks, wether the image is found somewhere in that screenshot (see how it works: http://sikulix-2014.readthedocs.io/en/latest/basicinfo.html#sikulix-how-does-it-find-images-on-the-screen)

So I do not understand your problem.

Revision history for this message
Gary Paschke (garypaschke) said :
#5

Thanks for all your time in helping resolve my confusion/understanding in this simple use case. I still am not getting results I would expect based on the simple script logic below.

If I run as is, I get "test passed", as expected. However, lets say, I add a small square markup (e.g. 10 x10pixel) on the 3D_Reciprocating_Saw_Gold.png with MS Paint and save. When I re-run script below, it still passed. If I exaggerate the square
to say 25x25pixel, it FAILED. I would expect ANY change to the saved 3D_Reciprocating_Saw_Gold.png to cause a failed test run, which is what we require. The test run (sikulix script) must match exactly what we are comparing on disk (i.e. 3D_Reciprocating_Saw_Gold.png)

image="D:\\Tc_Auto_Tests\\Images\\3D_Reciprocating_Saw_Gold.png"
r = Region(225,200,1675,810)
pimage = Pattern(image).exact()

if r.exists(pimage, 5):
   print "Screen matches 3D_Reciprocating_Saw_Gold, test passed"
   r.highlight(2)
else:
   print "Screen failed match of 3D_Reciprocating_Saw_Gold, test FAILED"

Any change I can upload an avi or word doc which may help?

Thanks

Revision history for this message
RaiMan (raimund-hocke) said :
#6

In Germany it is now evening. I will come back tomorrow

Revision history for this message
RaiMan (raimund-hocke) said :
#7

Read and understand:
http://sikulix-2014.readthedocs.io/en/latest/basicinfo.html#sikulix-how-does-it-find-images-on-the-screen

your problem probably is this:
Pattern().exact() internally checks, wether the match score is >0.99.
If you only make slight changes to the image, then this apparently still gets checked as matching.

You have to use a solution, where you check the match score yourself with all its decimals:

found = r.exists(pimage, 5)
if found and found.getScore() > 0.9999: # see comment
   print "Screen matches 3D_Reciprocating_Saw_Gold, test passed"
   r.highlight(2)
else:
   print "Screen failed match of 3D_Reciprocating_Saw_Gold, test FAILED"

comment: the challenge here is, to decide which value to use for the comparison decision match/not-matching (the reference value)

Revision history for this message
Gary Paschke (garypaschke) said :
#8

Excellent! Worked perfectly on my unit testing. Your response and help is exceptional .

Revision history for this message
RaiMan (raimund-hocke) said :
#9

Thanks for feedback.
Have fun with SikuliX ;-)