images with different resolutions or high similarity

Asked by leon

Have seen some threads on this topic, but based on previous Sikuli versions.

I am sorry if the question is duplicate, however, I really want to know, if images with different resolutions or high similarity can be recognized as same ones in SikuliX 1.1.0, or 1.1.0 which is under development.

If the answer is no, any good advice to improve? In my cases, for example, images on different phones, they look no different, I expect Sikuli think so too ;-)

I planned to introduce SIFT into SikuliX, but I read some reply posted by RaiMan which tells us Sikuli won't use SIFT in the future, and considering my limited knowledge to OpenCV, this is not probably good option for me.

In this link:
https://answers.launchpad.net/sikuli/+question/205371
Also see statement that external tool like ImageMagick's convert may solve this issue, not sure if SikuliX 1.1.0 supports it.

SikuliX is this great, I have to say another time, though many people said so many time. Thanks.

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
leon
Solved:
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said :
#1

This are 2 different things:

-- high similarity
This can and will be solved with version 2.
As one requirement for a solution, the images must concentrate on the distinguishing contents (as little background as possible towards the edges (this will be better supported in version 2 at time of snapshotting the image and there will be a possibility to automatically recapture an image set).
Another problem currently is the fact, that the similarity is already internally rounded to the second decimal, but image differences of only some pixels (e.g. different rendering of edges of graphic elements) might only be reported at a decimal far beyond the third decimal. This will be changed in version 2 too.

Another challenge with similarity valid from the very beginning of SikuliX is transparency. Since the images are rectangles, there might always be parts towards the edges, that are not relevant. Or testing people get images for testing from the development, that have transparent parts. Or one wants to find a button with changing text (e.g. localized). A transpareny channel in images currently is simply ignored, with affects, that are not generally predictable. This will be addressed in version 2 too (may be limited).

-- different resolution
Since SikuliX is based on matching the images by their pixel content, the image on the screen must have the same size (width and height) measured in pixels as reported by the underlying snapshot feature of Java Robot. If the image has a different size on the screen now, then SikuliX will not find it currently. You have to manually create a new image set for this environment.
For this situation there will be a solution in version 2, that allows to capture a new imageset on the fly by simply running a workflow that would fail with the original image set otherwise (this only works if the known positions of the images from a valid run are still the same may be evenly changed by a different resolution)
Another option, that already now works: If you know, how the images in the different resolution are shown, then you might on the fly resize your given images with this factor and might have success (if the rendering in the different resolution works such simple).
Features for that are available in version 1.1.0 in the class Image. Come back if interested.

-- SIFT
with version 2 I will again have a look at this complex but powerful feature and what it can help.
If you want to start with some experiments: I started with SikuliX version 2. Here I use the Java API available with OpenCV, which is a bit easier to use in situations where you do not want to deal with C++ code.
Come back if interested.

Revision history for this message
leon (xinli-huang2012) said :
#2

Thanks RaiMan. For now, can external tool like ImageMagick help us out in this situation?

Revision history for this message
RaiMan (raimund-hocke) said :
#3

What do you want to do?

Revision history for this message
leon (xinli-huang2012) said :
#4

I hope to use Sikuli to "scale down" image ratio while doing image comparison.
Is it possible to do that with ImageMagick?

Revision history for this message
RaiMan (raimund-hocke) said :
#5

yes you can do that with ImageMagick, but there are als other tools available, that allow to batch-resize image in a folder-tree being less complex.

Revision history for this message
leon (xinli-huang2012) said :
#6

Thanks RaiMan, that solved my question.

Revision history for this message
Lucas Ribeiro (lucasribeiro1990) said :
#7

Hey leon,

Can you help me explaining here hwo did you do that? (I'm new to this image recognition world)
I have the same problem, I think.
I have some tests automated for an application, but I have done this only in one resolution. Now the app has some machine dependency e I have to test in a different machine that has a lower resolution than the last one.

Revision history for this message
RaiMan (raimund-hocke) said :
#8

@Lucas
With version 1.1.0+ there is a not yet documented option, to resize an image with a SikuliX feature also:

img = Image.create("some_image.png")
print img
imgNew = Image(img.resize(factor)) # factor < 0 smaller, > 0 larger
print imgNew
imgNewFile = imgNew.asFile() # store in temp folder
print imgNewFile

... and this is possible then
if exists(imgNew):
    print "found"

Currently the feature internally used is the one of Java AWT for a BufferedImage.
In version 2 the OpenCV features will be used, which are more flexible with more options.