Unable to find images on identical monitors

Asked by Daniel Kurtz

Using v1.1.0. I have script running in Jython from the command line. Until recently we were executing this script successfully on virtual machine displays. Recently we've migrated from the VMs to physical desktop. All machines are identical and all are running on identical HP monitors. Resolution and color schemes are set the same.

The script runs fine on my system. It won't run on my co-worker's, despite everything being identical. Also despite the fact that it ran fine on his previous VM. Sikuli is unable to locate ANY image, returning a FindFailed. He has very methodically and painfully walked through and recaptured every single image in the script on his own machine, and those work fine. This obviously is not very practical, from a portability standpoint.

Why would it not run the same on two identical monitors displaying exactly the same thing?

Question information

Language:
English Edit question
Status:
Answered
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said :
#1

--- Why would it not run the same on two identical monitors displaying exactly the same thing?
You might have this impression, by just LOOKING at the images. But their MUST be differences on the pixel level, usually at the edges of contained graphic elements (anti-aliasing, font differences, ...).
You might check with an image viewer, that allows to look at the magnified images at the pixel level.

Take care, that your images concentrate on the graphic content, that makes it unique, and have as little background as possible towards the edges.

So you either manage, to have REALLY IDENTICAL environments with respect to the pixel level, or you need different image sets for different environments.

This situation will not change as long as SikuliX find feature uses OpenCV's matchTemplate(), but version 2 will have better support, to create a different image set on the fly when running a workflow, that does not work caused by these symptoms.

Revision history for this message
Daniel Kurtz (dkurtz-p) said :
#2

Thanks RaiMan. Compared images snapped on both systems and there are indeed slight differences in the way text is displayed... mostly in the choice of anti-aliasing colors, but there are also very slight variations in the shape of some letters. Nothing you can register by the eye at normal resolution.

I might play with lowering the pattern matching threshold but we've had difficulty with that. Often times the difference between matching just the item you want on this application and matching 10 items is just a percent or two.

d

Revision history for this message
RaiMan (raimund-hocke) said :
#3

Ok, that is what I expected.

Having text in the images to be searched for is a big problem for SikuliX, caused by the pixel-based approach, if you try to use the workflow in different environments (different rendering, different fonts and even different languages).

If possible, one should always try to search with patterns, that promise to be the same in different environments.
One approach is to search for one textless part of the image (even with findAll()) and then check the result with another textless part of the image, to get the match you are looking for.
A good example is a button, that has different text in different environments.

Another possible approach is to use relative coordinates as often as possible, which works in situations, where at least parts of the GUI is fixed according to the coordinates of the contained elements. Search one fixed element and act on others based on their offset.

Lowering the matching threshold might work in some situations, but always has the risk of false positives.

Hope you get your way.

Can you help with this problem?

Provide an answer of your own, or ask Daniel Kurtz for more information if necessary.

To post a message you must log in.