Click on text captured from Screen().text()

Asked by Hina on 2018-03-07

Hi,

I have got text using string1= Screen().text() function.
Can we extract coordinates of some sub-string ? and then by using these coordinates is it possible to click(or apply any operation)on the sub-string?

e.g. I have text on screen with button Yes.
Are you sure you want to continue?
Buttons:
No Yes
string1=Screen().text()
will return
Are you sure you want to continue?
No Yes

the simple OCR text search approach isn't successful in my application case :(

I want to click on Yes button without using image.

Question information

Language:
English Edit question
Status:
Solved
For:
Sikuli Edit question
Assignee:
No assignee Edit question
Solved by:
sami baig
Solved:
2018-09-07
Last query:
2018-09-07
Last reply:
2018-03-25
RaiMan (raimund-hocke) said : #1

you can try using
someRegion.find("some text")

which searches for a text and returns a match region, that can be clicked.

... but be aware of all the weaknesses of the text feature implementation.

Hina (hina-amir) said : #2

The region.find("text") won't work as expected.
I tried to print text from region and it displayed
"No Administrator account found " as "fldmin istrator account found "

If I search for Administrator it will throw Find failed error.
I tried with "account found" and result is same.
might be text feature weakness :(

RaiMan (raimund-hocke) said : #3

... might be text feature weakness :(
surely true

... but if you want to step into the gory details:
https://answers.launchpad.net/sikuli/+question/665345

Hina (hina-amir) said : #4

I used Screen().text() instead it gives you better results ... it had difficulty reading first letter of each sentence on CMD but otherwise it worked well. The only issue is I can't narrow down sear to specific coordinates on screen. It will take the monitor extents which makes my script slower. Let me know if ther's any way to change focus on particular the screen coordinates.

RaiMan (raimund-hocke) said : #5

instead of
Screen().text() (BTW: Screen(). not necessary - simply text() is enough for whole screen)

you might use
someRegion.text()

after before having evaluated someRegion with other SikuliX features
simpliest as
someRegion = Region(x, y, w, h)

or
someRegion = selectRegion() # interactive

or with the many other features, that aloow to define a region relative to other regions or locations on the screen.

Best sami baig (gabbarmic) said : #6

Hi,

Ap kahan rehti ho ?

Hina (hina-amir) said : #7

Thanks sami baig, that solved my question.

RaiMan (raimund-hocke) said : #8

How does a question:

Where do you live?

solve an issue like that?

@Hina
You should try the new 1.1.4 with the enhanced text features.