SikuliX

Feature request: Background removal in Sikuli

Asked by Tak Eda

Sikuli has issues with images that have non-white background color.

Sikuli will actually match completely different foreground objects that happen to have the same background, even if after removing the background color, they could be completely different images: http://imgur.com/a/IjsDk

Also, if the foreground objects are identical, any background color can result in less than ideal similarity score: http://imgur.com/a/9Yd9z

@RaiMan, how much work will be required to implement a find() function that implements background removal as in the following article:

http://answers.opencv.org/question/24463/how-to-remove-black-background-from-grabcut-output/

Also, if the Sikuli script designed was to pass a sample of the background to the find() function, will it make things easier?

Question information

Language:
English Edit question
Status:
Answered
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said : 		#1

@TakEda
Thanks for the pointer.

The basic problems with the current implementation of the image search:
- can only be a rectangle
- all pixels count (transparency is ignored)

saying it other way round:
you cannot use images for searching that are somehow "masked" to a visually non-rectangular shape or have some pixels (usually background to the edges) 100% transparent.

This option is on the list for version 1.2 with different solution paths:
- somehow take the transparent pixels into account during matching (as an option, makes search last longer)
- use edge detection to find a visual object while ignoring some inner parts of the image (e.g. buttons with different content)

So feel free to post more ideas on this. I will dive into this area during the next 3 months.

Can you help with this problem?

Provide an answer of your own, or ask Tak Eda for more information if necessary.

To post a message you must log in.