ignore transparent part of an image when searching

Asked by Patrick

Is there a way to notify sikuli to ignore some part of the reference image when searching for a match ?
The idea is to define a reference PNG to match a "container" but ignoring its content when the identification of the area on the screen and then to capture (image or text) this area. (E.g. having a transparent area in the reference image as a wildcard area)

Thanks

Question information

Language:
English Edit question
Status:
Answered
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said :
#1

sorry, currently not.

A possible solution:
- divide the "frame" into 2 or more images
- search with exists(partXofFrame, 0) for all image parts
- validate the matches, to be sure you got what you want
- create the resulting region from the matches

... can be packed into a function

A similar solution is possible with the observe feature.

Revision history for this message
Patrick (pguermo) said :
#2

I dont think the proposed solution can do properly the job as find could
return other areas than the piece of the one of concern. That's why I need
to be able to add a wildcard area (a mask on some parts of the area).
That's why a were thinking to use the transparency of the PNG files as
mask. Moreover, by now, using PNG files with transparency as reference
image doesn't work and I have to recreate new images by adding a background
on them.

I found a discussion on this topic on the opencv forum (
http://answers.opencv.org/question/53781/roi-always-a-rectangle/#53792)

"The ROI of a mat is not needed to be always a rectangle, but cv::Mat is
always a matrix, or a rectangle image, so you cannot have a round image,
but you may have a circle on a black background that contains the
information that you want. For having this, you have to use a maks:

cv::Mat image = cv::imread("img.jpg");
cv::Mat mask = cv::Mat::zeros(image.size(), CV_8UC1);
cv::Point circleCenter(mask.cols / 2, mask.rows / 2);int radius =
std::min(mask.cols, mask.rows);
cv::circle(mask, circleCenter, radius, CV_RGB(255, 255, 255));
cv::Mat imagePart = cv::Mat::zeros(image.size(), image.type());
image.copyTo(imagePart, mask);

2016-09-27 13:28 GMT+02:00 RaiMan <email address hidden>:

> Your question #402356 on Sikuli changed:
> https://answers.launchpad.net/sikuli/+question/402356
>
> Status: Open => Answered
>
> RaiMan proposed the following answer:
> sorry, currently not.
>
> A possible solution:
> - divide the "frame" into 2 or more images
> - search with exists(partXofFrame, 0) for all image parts
> - validate the matches, to be sure you got what you want
> - create the resulting region from the matches
>
> ... can be packed into a function
>
> A similar solution is possible with the observe feature.
>
> --
> If this answers your question, please go to the following page to let us
> know that it is solved:
> https://answers.launchpad.net/sikuli/+question/402356/+confirm?answer_id=0
>
> If you still need help, you can reply to this email or go to the
> following page to enter your feedback:
> https://answers.launchpad.net/sikuli/+question/402356
>
> You received this question notification because you asked the question.
>

Revision history for this message
RaiMan (raimund-hocke) said :
#3

--- I dont think the proposed solution can do properly the job ...
Feel free to not test it ;-)
I know, that other SikuliX users have done this.

--- I found a discussion on this topic on the opencv forum ...
... having masked images is not the solution.
The challenge is, to either teach the matchTemplate() feature (the feature used by SikuliX), to accept and honor transparent areas in a given image or use another feature to search for one image in another with a duration of some milliseconds (e.g. edge detection).
Both approaches are are on the list for version 2.

conclusion: see my comment #1 ;-)

Can you help with this problem?

Provide an answer of your own, or ask Patrick for more information if necessary.

To post a message you must log in.