How to improve the image quality generated from PIL when compared to image captured by Sikuli?

Asked by srdat on 2017-10-09

What I'm doing is as below -

My line of code in sikuli IDE as below

testlibrary.click('ExcelDocument')

I read the text(ExcelDocument) passed to click from the above line, generate a png image with the same text using segoqui.ttf with size of 11 and store it in image library and finally perform a click.

It works for ExcelDocument with default similarity
--
Issue:
testlibrary.click('MS Word Document')

The same code generates image but fails to click with default similarity but works with lesser similarity which is not good and this is impacting our progress.
--
When I compare the images I generated using my code with the ones capture by Sikuli tool, i see a difference with respect to quality with the edges of letters are slightly blurred and also few letters has a tiny space between them. Is that causing a problem?

How do I overcome this and generate a good quality of image from text?

Question information

Language:
English Edit question
Status:
Answered
For:
Sikuli Edit question
Assignee:
No assignee Edit question
Last query:
2017-10-09
Last reply:
2017-10-18
TestMechanic (ndinev) said : #1

I think you are wrongly assuming that your lib and application under test has same fonts and same rendering mechanism.

Maybe there is another way to achieve your goal

srdat (srdat) said : #2

yes, you are ccorrect!
The image I generated from my lib is little different(at a very tiny level) than the application though I have used the same ttf. I think it has to be fixed in my python library to create an exact image. I'm still thinking of how to achieve it.

One way to generate text to be recognized would be to type all possible characters in the software you will use, capture and crop the characters individually and regenerate the text you want to find from these characters using some custom kerning rules to control the space between the characters. This method works but takes a lot of work and is very intolerant to changes in the way the text is rendered in the screen.

RaiMan (raimund-hocke) said : #4

If you are not satisfied with SikuliX's 1.1.x OCR quality and timing is not really a problem:
Install Tesseract 3.x and use the tesseract ocr command internally (input image - output text).
advantage: all possible options are available and you have the font learning features.

srdat (srdat) said : #5

I tried tesseract as per your suggestion but here also I'm not seeing a good results. It clicks for few texts and fails for few. If I reduce the similarity it works but it is not recomended

#below is my code
def TextSearch(image,TexttoSearch): #here image is referring to the screen capture of my screen in png format where all the required texts to be clicked are present
    reg=define_reg_full()
    for i in (TexttoSearch):
        sleep(1)
        print i
        try:
           label=reg.find(i)
           label.rightAt().click()
        except FindFailed:
            print i +" find failed"
        sleep(1)

TexttoSearch=["myDocument", "mypresentation19-12.pptx", "doc-1239342a", "testuser","version0.1"]
TextSearch("1507054309107.png",TexttoSearch)

In the above array, click happens only for myDocument & testuser and for remaining it fails. Any suggestions?

RaiMan (raimund-hocke) said : #6

might be, that my comment #4 was not clear enough:
I suggested to use the Tesseract 3.x package outside of SikuliX after having it installed and made your tests from command line.
If these are ok, then you can think of integrating the command execution using subprocess.popen.

The SikuliX internal OCR features are bound to Tesseract 2.x library and have the weaknesses you encountered.

srdat (srdat) said : #7

I tried to read the text from the image captured using tesseract outside sikulix, it identified few texts from the image and few it didn't.

Please let me know if this is what you suggested to try or there is a different way around?

RaiMan (raimund-hocke) said : #8

if you used a Tesseract 3+ installation with the Tesseract command on a command line - then yes, this was my suggestion.

To optimize your results, you have to take try the different options as documented with Tesseract.

This is all, I can do for you.

Can you help with this problem?

Provide an answer of your own, or ask srdat for more information if necessary.

To post a message you must log in.