SikuliX

OCR Tweaking

Asked by chyen

Thank you for taking time reading my question.
First of all, I m not a coder therefore I m not so good when passing the parameter around. Please enlighten me if I m wrong.

I have a screen to scan, the image is clean with digits only appeared on the screen but the problem is the text is in white color with grayish background which makes the entire image rather low contrast.

I already tried playing with various setting such as OEM, PSM, font size, height, digits only config ++ etc but has no luck to extract the digits with high accuracy. The accuracy is only around 50%. I realized that I need to tweak the image before passing to OCR, by making it B&W can extract the font with good accuracy (Tested it with tesseract). https://docs.opencv.org/master/d7/d4d/tutorial_py_thresholding.html

I also know that with Sikulix 2.0.3 that i m using now I may access to the OpenCV library such as org.opencv.imgproc to do the image processing manually and read the output image with OCR.readWord(image,options). However now I m stuck at the image and material(Mat) conversion since openCV rely on the Mat to analyse the image. And I also got to know that numpy cannot be brought in due to the language restriction.

So now I m kinda stuck here and have no idea what can I do to improve further. I don't think I need to generate traineddata since my source was just uniform digits. Anyone has any idea what or how can I tweak more? OpenCV or something else that I m not aware of? Thanks.

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
RaiMan
Solved:
Last query:
Last reply:
Revision history for this message
Best RaiMan (raimund-hocke) said : 		#1

First of all: please use 2.0.4, so I can help you better.

Second: please send me a sample of such a number image to my mail sikulix---at---outlook---dot--com (original as shown on screen)

When using the OCR features of SikuliX the image is optimised according to the recommendations, before giving it down to Tesseract.

I have to admit, that you are talking about an edge case, where the conversion to grayscale as part of the optimisation, is not sufficient and black/white is needed.

I will add such an option to version 2.0.5.

With the Mat-conversion I cannot help you in the moment, since the respective API, that is used inside of SikuliX, is not accessible from the outside world (non-public).
To support such experiments I will make the features image-to-Mat and Mat-to-image available in 2.0.5.

You might look at the code and rewrite it in a Jython script, since you can use Java in this environment.

--- Image-to-Mat
org.sikuli.script.Finder.Finder2.makeMat(BufferedImage bImg, boolean asBGR)

--- Mat-to-image (as .png)
org.sikuli.script.Finder.Finder2.getBufferedImage(Mat mat)

Revision history for this message
chyen (chyen) said : 		#2

Thanks Raiman! Good to hear that the OCR features is getting enhanced in the future build!

I will to figure out the Jython stuff and give it a try with your suggested lines while awaiting your next build. Appreciated.

Revision history for this message
chyen (chyen) said : 		#3

Thanks Raimund and 2.0.5 solved my issue wonderfully!

Revision history for this message
chyen (chyen) said : 		#4

Thanks RaiMan, that solved my question.