SikuliX

Is the integration of Sikuli with openCV/Tesseract totally removed in the newer version ?

Asked by Sai Nikhil

I was browsing through the project present at this (https://github.com/RaiMan/SikuliX-Basics) location. In package org.sikuli.basics.proxies, I can see all the files related to OCR and openCV, which were .cpp in previous version (https://github.com/sikuli/sikuli/tree/develop/sikuli-script/src/main/native).

Does this mean that natives in sikuli are now written entirely in java, which makes it free from C++ (and so, openCV and Tesseract too) Please correct me if I'm wrong somewhere.

If my belief is true, why is that particular file in the package (org.sikuli.basics.proxies) named openCV.java, even though its free from openCV ? Is it because of the equivalence ?

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
RaiMan
Solved:
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said : 		#1

--1. f you reference github, then the top project is either
https://github.com/RaiMan/SikuliX-IDE
or
https://github.com/RaiMan/SikuliX-API

--2. If you want to use the new version, start here:
http://www.sikuli.org/download.html

--3. about OCR/textSearch
- look https://answers.launchpad.net/sikuli/+question/232900
... and the IDE preferences
... and faq 2436

the OCR/text feature is still at level X-1.0rc3 with some minor improvements.

Revision history for this message
RaiMan (raimund-hocke) said : 		#2

BTW: what you see in org.sikuli.basics.proxies
are the SWIG generated interface classes to call the native modules via JNI
The sources of the native modules to integrate OpenCV and Tesseract are at
https://github.com/RaiMan/SikuliX-VisionProxy

Revision history for this message
Sai Nikhil (saint1729) said : 		#3

Hi RaiMan,

Thanks for your reply. I started using sikuli from the download link you gave above. Now, I want to build this sikuli-ide.jar following your instructions at github. First of all I wanted to clear some of my basic doubts regarding recent changes happened in development of Sikuli. Is sikuli now free from openCV and Tesseract ? Is it entirely a java project now ?

Revision history for this message
Sai Nikhil (saint1729) said : 		#4

Hi RaiMan,

In reply to your second message:

So, firstly do I need to generate interface classes from https://github.com/RaiMan/SikuliX-VisionProxy using SWIG ? Can you please post the instructions on github on how to do this and also how to integrate these with SikuliX-Basic ?

Revision history for this message
Sai Nikhil (saint1729) said : 		#5

Btw, I use windows 7.

Revision history for this message
RaiMan (raimund-hocke) said : 		#6

As stated on the projects page:

It is contained in all downloadable packages and is not intended for any usages outside these packages.

Forking only makes sense, if you want to contribute features or patches.
Use mvn install in the project folder to get it in your local repo.

So what is your intention?

Revision history for this message
RaiMan (raimund-hocke) said : 		#7

try to build the stuff yourself:
starting here
https://github.com/RaiMan/SikuliX-IDE

you have every info you need

Revision history for this message
Sai Nikhil (saint1729) said : 		#8

Hi RaiMan,

My prior intention is to edit the C++ files at the native level to improve accuracy of OCR, which is straight forward if I'm with project SikuliX-VisionProxy. Since I'm new to java world, I wanted to know how to integrate my changes done to C++ with the final .jar that comes out.

Revision history for this message
Best RaiMan (raimund-hocke) said : 		#9

Ok, getting the VisionProxy.dll built on Windows is a stony way.

Since I do not have any experience with the DevChain on Windows, I decided to use minGW.

To produce the currently published VisionProxy.dll, I use a Netbeans C++ project, where I have built OpenCV and Tesseract from the sources using the minGW environment and the make procedures of the mentioned packages.

This now works for me, to get distributable VisionProxy.dll for 32-Bit and 64-Bit.
I plan to visit the native stuff again in preparation for version 1.1 later in November.

If you are a somewhat professional C++ programmer, in the moment it would help a lot to have a robust build approach based on the latest versions of OpenCV and Tesseract using the freely available tools of the Windows DevChain with a step-by-step how to.

My plan is to later divide the native stuff into the 2 major parts:
VisionProxy.dll - everything related to OpenCV (currently in fact it is "only" matchTemplate() )
TextProxy.dll - for all the features related to textSearch and OCR (I have already added findText() methods internally)

So if you start bottom-up, you might do what you want, as long as the interface up to the Java level stays stable.

You even might restrict your work to the C++ level and make a small wrapper in C++, that simulates the Java stuff around in the sense, that you fill the findInput structure accordingly and call find -- should be rather simple.
... and should be far better debuggable, since you stay in the C++ environment.

But first of all you have to setup your DevChain together with suitable packages of OpenCV and Tesseract.
If you are then able, to make you own VisionProxy.dll, this will work with the currently available packages (no need to step up to the java level or use SWIG, as long as the interface to the Java level stays stable).

Come back whenever you want.

Revision history for this message
RaiMan (raimund-hocke) said : 		#10

one more thing:
having a project on github (e.g. SikuliX TextProxy) would make it easy, to work together.

Revision history for this message
Sai Nikhil (saint1729) said : 		#11

Great Explanation. Thank You very much. I'll try building VisionProxy.dll in my Visual Studio right now and will update you.

Revision history for this message
RaiMan (raimund-hocke) said : 		#12

Thanks for feedback.

Visual Studio is fine.
... but what you do should be reproducible with the freely available stuff from MS

Revision history for this message
Sai Nikhil (saint1729) said : 		#13

Yeah, I know the terms of OpenSource ;)

Revision history for this message
Sai Nikhil (saint1729) said : 		#14

Thanks RaiMan, that solved my question.

Revision history for this message
Sai Nikhil (saint1729) said : 		#15

Hi RaiMan,

Small doubt. How many of the libs in this (https://github.com/RaiMan/SikuliX-Basics/tree/master/src/main/resources-libs/libs/windows/libs32) are strictly required for the functioning of Sikuli-IDE. I'm asking this question because I presumed that for building VisionProxy.dll I need opencv, tesseract, libpng, leptonica as pre-requisites and after it is built, I no longer need the later ones. All I need is VisionProxy.dll. This is just my assumption. Please correct if I'm wrong somewhere.

Revision history for this message
RaiMan (raimund-hocke) said : 		#16

VisionProxy.dll was linked dynamic from the beginning, so I continued to do this without really checking, wether this would be the best or only possible approach. With dynamic linking the dependant libs have to be available at runtime.

I never tried to link VisionProxy.dll as a static library, in which case the mentioned dependant libs would only be needed at build time as far as I know.

So to ask your question:
Currently all the mentioned libs are only needed by VisionProxy.dll due to dynamic linking. These libs are not needed by any other part of Sikuli.