Differing images matched with very high similarity

Setting the default similarity to 0.98 is high risk.
You have to have very good snapshots: as little surrounding even background as possible and hence concentrating on the visual aspects making the difference. Version 1.2 will have a feature to optimize snapshots in this direction.
But till then, you have to try to get optimal images yourself.
This is surely the problem with your gray image.

So with the current implementation a practical value to be used for default similarity is 0.95, when your workflow expects exact matches and you allow your snapshots to be slightly less than optimal.

Especially black and white areas as background in the current implementation still may lead to false positives, if the number of background pixels exceed 10 to 20% of the total amount of pixels in the image.
Here currently the only possibility is to concentrate on the non-black/non-white areas.
in your case the areas where the symbols appear.
Version 1.2 will have better support for these situations.

Thanks for the quick response RaiMan - to clarify, that our current issue is not false negatives, as we have an easy mechanism to update our reference imageswhen there is a change, but the false positives, which are much harder to detect - requiring manual analysis of the test logs.
Could your be more specific in your remark about version 1.2 - is there going to be a change in the general image comparison algorithm, because if I understand correctly - it is based on OpenCV and is out of the scope of Sikuli development?
Our current best solution is to put manually even higher similarity on reference images that have high black (also white it seems) content.
Forgot to mention, that we are using the Java API and not the IDE.

Thanks RaiMan, that solved my question.

Just to add, that it would be really nice if the Pattern java classs can be extended with more advanced control of the used OpenCV matching method and its parameters. Do you think it is feasible and usefull?

--- Do you think it is feasible and useful?
Might be. I will find out and add respective features during the next months.

Thanks.

as you might have seen already: made it a request bug

Sikuli has issues with images that have non-white background color

Just as you have observed, Sikuli will actually match completely different foreground objects that happen to have the same background, even if after removing the background color, they could be completely different images: http://imgur.com/a/IjsDk

@RaiMan, how much work will be required to implement a find() function that implements background removal as in the following article:

http://answers.opencv.org/question/24463/how-to-remove-black-background-from-grabcut-output/

Also, if the Sikuli script designed was to pass a sample of the background to the find() function, will it make things easier?

@Boris, I have created a new question (https://answers.launchpad.net/sikuli/+question/256255) from yours to better focus on this. Let's take the discussion forward there. This is different from the one you already have here at https://answers.launchpad.net/sikuli/+question/256236

Feel free to refer to this if you create a "request bug".

@TakEda
Thanks for the pointer.

The basic problems with the current implementation of the image search:
- can only be a rectangle
- all pixels count (transparency is ignored)

saying it other way round:
you cannot use images for searching that are somehow "masked" to a visually non-rectangular shape or have some pixels (usually background to the edges) 100% transparent.

This option is on the list for version 1.2 with different solution paths:
- somehow take the transparent pixels into account during matching (as an option, makes search last longer)
- use edge detection to find a visual object while ignoring some inner parts of the image (e.g. buttons with different content)

So feel free to post more ideas on this. I will dive into this area during the next 3 months.

I will add this comment to your request answer as well.

How do you display the actual similarity between your two images?

@Justin
I do not understand you question in this context.

Boris stated that 'The following two black images, for example, are matched with 0.982491' He also says: 'sometimes exceeding our default level of 0.98'.

I am just wondering how Boris knows the exact similarity value if the test outcomes(eg 0.982491). What would the code for seeing the similarity value look like?

@Jus
Each image comparison / image search results in a so called Match, that contains the top left corner, where the found image is located in the base image or on the screen, besides some other information.

On of these values is the score, a value between 0 and 1, denoting the probability, that the match is the searched image.
0 means no similarity at all, whereas 1 means should be the image exactly.

might be of interest: http://www.sikulix.com/stories/how-does-sikuli-find-images-on-the-screen

in a script saying:

m = find(someImage)
print m.getScore()

would reveal the resulting value of the match.

Thanks for the reply, I understand now.

I am having issues where I take a screenshot of the last screen of a test and try to run a new test to see if that screenshot is in my test. I am getting only 60% similarity though.

For example, I am filling in login information for GMail.

Goto google.com
Click gmail
Enter an email
Click Next

I take one screenshot of the middle(including Google, One Account. All of Google and the login form/buttons).

I run a new test to see if this screenshot exists, and I get a 60% match. I thought it would be closer to 99%.

Any ideas? I can paste code in Java if you'd want it.

@Justin
I just made some quick tests (with 1.1.0)
I always get score values > 0.9999...

if you send me the images/screenshots zipped together to my mail rmhdevelop---at---me.com, I can have a look at it.
The code you use is not really needed.

I think my issue may have been that I wasn't adding enough of a time delay in between the actions and the test. I have added a delay and I am getting 100% similarity now.

If a webpage is not ready instantly and completed visually overcome time in steps, you might get a found with low similarity.

This can either be compensated by adding some waiting time before the search, or by creating a Pattern with the image and a similarity of 0.95, which would internally wait for screen content that satisfies the condition.

I prefer the second solution.

problematic code:
click(button)
if (exists(Pattern(image).similar(0.5))

solution 1:
click(button)
wait(2)
if (exists(Pattern(image).similar(0.5))

solution 2:
click(button)
if (exists(Pattern(image).similar(0.95), 5)

this would give the search max 5 seconds time to find with a high score, but the search would succeed already after 1 second if found with the given similarity. So this is more dynamic and leads to faster scripts in the average.