SikuliX

Questions on functional capabilities of Sikuli

Asked by Ozgur Ibek

Hello,

We have been considering using this tool in our company. However before we decide there are a couple of questions in mind. I'd appreciate it if you could help me with these questions.

1- Does the tool support non-English characters, esp. Turkish?
2- How well is the tool's backward compatibility? (Opening and executing scripts developed in old versions) (Will be more important once the new versions keep coming)
3- What image recognition algorithms are supported?
4- Is it possible to extend (improve) image recognition algorithms?
5- Is there any mechanism for image recognition failure mitigation?

If you could provide detailed information about these questions that woud be fantastic.

Thank you very much.

Regards.

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
RaiMan
Solved:
Last query:
Last reply:
Revision history for this message
Eugene S (shragovich) said : 		#1

Regarding all your text recognition questions

Sikuli is using Tesseract OCR which is a well known OCR engine. It has its limitation and in case of working in a bundle with Sikuli still requires some work.
You can still work directly with Tesseract to try to improve the quality of recognition. You can find plenty of articles and tutorials on that topic.
Actually for all your question regarding language support and recognition improvement refer to Tesseract documentation directly.

Regarding image recognition
Sikuli's image matching feature is based on OpenCV's matchTemplate() (you can see the exact algorithm here: http://docs.opencv.org/modules/imgproc/doc/object_detection.html).
If you have another efficient image matching algorithm that might be implemented, feel free to do so.

Not so sure what you meant by "image recognition failure mitigation"

Revision history for this message
Ozgur Ibek (oibek) said : 		#2

Thank you for your reply.

As far as I can understand, not only I can manipulate the image recognition code but also I can set the system to use another algorithm, is that correct?

What I meant by "image recognition failure mitigation" is that when image recognition fails for a specific image, does the system immediately throw an error or first try something else? In an article an example was given as "to add redundancies in the test script code such that if the image recognition fails, it is automatically rerun or executed with another sought image."

Revision history for this message
RaiMan (raimund-hocke) said : 		#3

--- As far as I can understand, not only I can manipulate the image recognition code but also I can set the system to use another algorithm, is that correct?

no, that is not correct: the "image recognition code" is fixed by the implementation. Am image search with a given image either succeeds (found) or fails (not found) with the given minimal wanted similarity.
About the magic behind the scenes read:
http://sikulix-2014.readthedocs.org/en/latest/basicinfo.html#sikulix-how-does-it-find-images-on-the-screen

--- image recognition failure mitigation
the image search functions in the API either throw an exception (wait, find) or return a null match (exists). For the actions in case of not found you are completely responsible yourself by providing code, to handle these situations.
The implementation currently does nothing else, then repeatedly retry the search until the given maximum wait time is reached.

Revision history for this message
Best RaiMan (raimund-hocke) said : 		#4

regarding the questions 1 - 5 (based on version 1.1.0+)

at1: unicode/utf-8/non-English character support is given by the used scripting environment (Jython, JRuby, JavaScript) and in case of Java programming by Java itself.
regarding text recognition see comment #1

at 2: versions 1.1.0+ will be backward compatible either implicitly (simply works) or explicitly by using compatibility features.

at 3 to 5: see comment #3

Revision history for this message
Ozgur Ibek (oibek) said : 		#5

Thanks RaiMan, that solved my question.