SikuliX

Modify Vision.cpp?

Asked by James

I've been using Sikuli for a school project recently and am very impressed with the program. The issue I've been running into though is that the current vision algorithms are insufficient when either 1) background changes 2) transparency is enabled. There are of course easier ways to solve this problem ( increase training set), but being a student I prefer to get my hands dirty so to speak. So I've been digging into SikuliX and found that the vision algorithms are called using JNI to OpenCV and to vision.h.

Is it possible to modify the vision.cpp implementation to include simple sobel edge detection to improve the detection process?
If so then do I build the sikuli libraries again with the make files located in StuffContainer/LibsBuild/build/windows/TheBuildProcess/11_Vision/ ?

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
RaiMan
Solved:
Last query:
Last reply:
Revision history for this message
Best RaiMan (raimund-hocke) said : 		#1

of course is that possible, but I would not recommend, to step down to the C++ level and especially not to try to modify the C++ code of SikuliX (believe my experience: it is good for nightmares ;-)

It is much easier, to stay on the Java level and use the OpenCV Java API, especially, if you want to implement an image search based on edge detection (is on my list for version 2).

BTW: At least for the situations, where background changes and transparency are mainly towards the edges, it normally helps, to make the image as small as possible by leaving out as much of this stuff as possible. My experience: if the areas in an image, that might change or are transparent are below 20 - 10% pixels of the total pixels in the image, the search usually works sufficiently (scores above 0.9 - 0.95).

Examples for how to use the OpenCV Java API you can find in SikuliX version 1.1.1 (class ImageFinder where I rewrote the matchTemplate based search on the Java level) and of course in SikuliX version 2 (currently only file handling and CVMat).

If you decide to try on the C++ level, come back for more instructions.

Revision history for this message
James (hamsession) said : 		#2

Thanks RaiMan, that solved my question.