Recognize text (OCR) and paste it to a document?

Asked by tjonnyc

Here's what I would like to accomplish:

1.) Select a region on the screen.
2.) Do OCR on the text within that region.
3.) Switch to a different tab in the browser.
4.) Click on a specific location.
5.) Paste the recognized text.

I know how to do 1, 3, and 4.

Could someone help me with steps 2 and 5?

Region(1041,194,250,123)
{recognize OCR text somehow} <------ NEED HELP
click(Location(170,300)) #Click outside of Flash to regain focus
wait(1)
type("1", KeyModifier.CTRL) #Chrome shortcut - go to 1st tab
click(Location(80,280)) #Cell A2
{paste the recognized text somehow} <---- NEED HELP

Question information

Language:
English Edit question
Status:
Answered
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Eugene S (shragovich) said :
#1

You can do this like that:

Settings.OcrTextRead = True #set the OcrTextRead setting to True
text = myRegion.text() #recognize the text whithin the region and save it to a variable

Then you can just the resulting 'text' variable for your purposes.

Revision history for this message
Eugene S (shragovich) said :
#2

P.S. However you should notice that the text recognition functionality is not very stable. So before you commit any effort, you should first see how well it performs..
For that just print on the screen whatever was captured before and see if whatever was captured is usable for you.
Sometimes, if just certain characters are not being recognized correctly, you can create some auxiliary function to normalize the resulting text.

Revision history for this message
tjonnyc (tjonnyc) said :
#3

Thank you for the answer.
However, it's not working out.

var_name = myRegion.text()

Fails with "NameError: name 'myRegion' is not defined"

If I change it from myRegion to Region, I get the following:

var_name = Region.text()

Fails with "TypeError: text(): expected 1 args; got 0"

Revision history for this message
RaiMan (raimund-hocke) said :
#4

text() is an object method, so you need a Region object, that you can ask for the contained text.

easiest way to create a Region object interactively:

reg = selectRegion()
print reg.text()

run this in the IDE and you will be able to select your region and hopefully get printed the contained text in the message area.

in a real script you would create a region on the fly with one of the many available features.

Revision history for this message
RaiMan (raimund-hocke) said :
#5

BTW: when using 1.0.1: did you select the Tesseract option (5) at setup?

Revision history for this message
tjonnyc (tjonnyc) said :
#6

1.) OK, that got rid of the errors on recognizing the text...

The last issue is, what's the correct syntax for pasting the text?

2.) Re Tesseract selection: I don't remember, but I think I've downloaded the "All-in-one" package. Is there a way to check if Tesseract is installed?

Revision history for this message
RaiMan (raimund-hocke) said :
#7

at 1:
if the Gui is already positioned at an element that can be pasted in
paste(text)

or
paste(some_image, text)
if you first need to click on some_image to get the fucus to the right input field.

at 2:
forget about it: you have it, since reg.text() seems to work
(there is a folder tessdata in the folder libs in the installation folder)

Revision history for this message
tjonnyc (tjonnyc) said :
#8

The code:

reg = selectRegion(Region(1364,281,75,18))
print reg.text()

Produces the following error:

[info] Sikuli vision engine loaded.

[info] VDictProxy loaded.

[error] Stopped
[error] An error occurs at line 1
[error] Error message: Traceback (most recent call last):
 File "C:\Windows\TEMP\sikuli-tmp5050479256018944252.py", line 1, in
 reg = selectRegion(Region(1364,281,75,18))
 File "C:\Program Files (x86)\Sikuli X\sikuli-script.jar\Lib\sikuli\Sikuli.py", line 142, in selectRegion
TypeError: selectRegion(): 1st arg can't be coerced to String

Revision history for this message
RaiMan (raimund-hocke) said :
#9

selectRegion() is an interactive function, that expects a string to display in the overlay window used to create the region visually (like with capturing an image). This is the function behind the top left third button in IDE.

to define a region with absolute coordinates simply:

reg = Region(1364,281,75,18)

Revision history for this message
Pramendra (pramendra-ursfrnd) said :
#10

Hi RaiMan
I am facing one issue. I have written below code.

 if i have not saved it, i am getting expected result in "mylog.txt", even i am running multiple times.
But as soon as i saved it, for the first time i am getting correct output in "mylog.txt", and 2nd time onwards, unexpected things getting copied to "mylog.txt", Why so? What is the issue or What i am doing wrong here.

Below logic has been written in Sikuli GUI:

Settings.OcrTextRead = True
reg = Region(16,57,528,24) ## 16,57,528,24 co-ordinatest you will get from "Create Region" Option
score = reg.text()
print score

f=open("C:\mylog.txt" , "w+")
f.write(score)
f.close()

Revision history for this message
Pramendra (pramendra-ursfrnd) said :
#11

After giving some delay, the code is working fine for multiple times even after saving.
May we little processing time, we have to give for OCR to work.
I got the solution so you can skip to answer. Any suggestion/info you can share , i am glad.
Thanks.

Can you help with this problem?

Provide an answer of your own, or ask tjonnyc for more information if necessary.

To post a message you must log in.