Sikuli and Vector Images --- does not matter - only pixels are relevant

Asked by Rajesh

Hi,

Using Sikuli i capture a vector image(template) from a system having resolution of 2560 x 1600 . If i try to run the same script in lower resolution say 1440 x 900, then i get a find failed error.

Now using Image magic if i resize the template image to 66% (convert template.png -resize 66% template.png) then in lower resolution system, sikuli is able to find the image with a matching of 0.95

I wanted to understand how this is working .

Thanks in Advance for your help
Rajesh

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said :
#1

Sikuli only sees pixels on the screen and does not know anything about how the image was rendered to the screen.
A basic requirement for Sikuli to find an image is that the visible image has the same size in pixels as the image we are using to search. If this is not the case, then you get a FindFailed.
The similarity score only tells you about how similar the sum of pixels are (which is some statistical value).

So if some app renders images differently in different environments (like in your case), the image will not be found.

In your case you adjusted the image correctly to the new situation and hence it was found.

We have a similar problem on Macs with Retina displays.

It is on the list for enhancements, to better support these situations (using the same image successfully in different environments).

Revision history for this message
Rajesh (rajesh-thekkadath) said :
#2

"It is on the list for enhancements, to better support these situations (using the same image successfully in different environments)"

Looking forward to it. Thank you !

Revision history for this message
Rajesh (rajesh-thekkadath) said :
#3

Raiman,

Is it possible you can discuss on how you plan to implement to support this feature (high level) or if there is already some discussion documented else where , can you point me to that? Does it need a big architectural change in Sikuli to incorporate this feature?

We are planning to implement a large scale automation for testing flash based animations using Sikuli and this is must have for that. Our requirement is to support testing of the same animation in different resolution from 800x to say 2500x. The animations are vector so it resizes itself in multiple resolution . The image stays the same

Till the feature is implemented in sikuli, is it wise to write a image transformation layer which will resize images dynamically by factor based on the system resolution and find a match ? (i have seen eggplant doing something similar http://docs.testplant.com/?q=content/finding-images#scaling-by-a-factor-anchor)

Below is our high level design

1. 1000's of independent sikuli script ---> 2. Transformed to use Sikuli API + Webdriver and runs in multiple browser and android
(only has click, wait and dragDrop)

Revision history for this message
RaiMan (raimund-hocke) said :
#4

--1. the feature is not yet planned

--2. is it possible, to get samples, that I can use in a browser, to force different rescaling and check, how the search will work, with resized versions of some base image?

--3. a basic implementation with minimal effort to implement:
factor = <some evaluation/calculation resulting in a value 0 < 1.0 < max>
img = Pattern(some_base_image).resize(factor)
find(img)

The "problem": since the probe is only found, if width and height of the image in the current visible resolution is the same as the probe, we have to check some range of width around the calculated width base on the given resize factor.

example:
probe width = 105 pixels
resize factor: 1.237
calculated new width: 129.885

so we might try the following strategy:
check with resized image having width:
130
129
131
128
132
127
133
126
...
until found.

with some setting, the range around the calculated value should be restricted.

To avoid lengthy processing with this feature, it should be possible, to once at the beginning of the workflow to find out with some reference image, what the optimal resizing factor is, so that in the above example only a few width values around 130 and 129 must be checked.
The calculations can further be supported, since with 1.1.0+ it is possible to get the dimensions of a loaded image for calculations.

This would make another option possible:
newWidth = <some evaluation/calculation resulting in the int pixel width of the image in the current environment>
img = Pattern(some_base_image).width(newWidth)
find(img)

So the workflow is responsible to set the new width, to that the probe should be resized.
So only one search is needed.

I could make this feature available in the current Beta within a week or so, since internally, the needed base functions are already available.

If you are interested, then we should communicate privately from now on: use my mail at https://launchpad.net/~raimund-hocke

Revision history for this message
RaiMan (raimund-hocke) said :
#5

There is another feature, that I have already planned: image groups

So in your case it might be possible to say, all images that have to be resized belong to an image group named images1.

g1 = new ImageGroup("images1", <path to folder containing the images>)
g1.resize(factor) or g1.width(newWidth); // to be implemented according to the mentioned Pattern feature
// so every image in that group will be processed before used in a search
reg.find("images1/some_image");

This image group feature is mainly targeted to the cases, where you need different images for different system/browser/application environments. So you might collect these images in different folders and decide at runtime, which folder to use, by defining the group before accordingly.

Revision history for this message
Rajesh (rajesh-thekkadath) said :
#6

Raiman, You are super cool !!! I will continue the discussion privately.

Revision history for this message
RaiMan (raimund-hocke) said :
#7

made it a request bug

Revision history for this message
Tak Eda (takeda) said :
#8

Raiman, your suggestion #4, is excellent!

Would adding such a parameter be too much work?

I can ry and help too, but would love to see this feature.