Sikuli X 1.0rc1 - How to get an exact match with click("String")?

my experiences with the text feature now:
- if find("some-text") returns a match at all, it is always reported with a score of 1.00
- the match might not be the text, you are looking for (anyway 1.00 is reported)
  (e.g. "Create", "create", "xreate", ... all find the same text "Create")
- you may try to get the text from the returned region and compare it with your search text

r = someRegion
t = "some-text"
m = r.exists(t, 0)
if m:
    m.highlight(2)
    if m.text() == t:
       print "exact match"
    else:
       print "found something else"
else:
    print "nothing found at all"

I think, in the current still experimental state of the text feature, you have to setup some helpers, to be sure you get what you want.

One more thing:
I found, that (as used above) reading the text from a text match region does not return the correct text. It seems that especially uppercase letters are often mis-read.
using instead
m.nearby(1).text()
normally returns the correct text. This might be a bug.

You should make your own experiences and talk about it ;-)

Actual text on the screen(integer in this case) => 534438
Sikuli X OCR recognise it as => 534433
which mean that '8' is interpreted as '3'

Actual text on the screen(integer in this case) => 534177
Sikuli X OCR recognise it as => 5341??
which mean that '7' is interpreted as '?'

Here's the Text on the screen (http://qoydkw.bay.livefilestore.com/y1pSr0oDju9ndvMzJJfo_qWD2qEvMdDDwoKquMNWWkUYAxgrsVcXmETY8yW_LqIieOKXgwlTt8yxe8eQDY-SCySCFljo59mSSIC/TextOnTheScreen.png?psid=1)

--------------------------------------------------------
r = Region(“TextOnTheScreen”)
t1 = "534438"
t2 = "534177"
t3 = "4438"
t = t2
m = r.exists(t,0)
if m:
 if m.text() == t:
  popup("Exact match")
 else:
  popup(m.nearby().text())
else:
 popup("Nothing found at all!")

I confirm, that this is a general problem: same on my Mac!

You could report this as a bug. I will report the fact, that matched-region.text() normally does not return the correct text.

But: test it with the text in your comment above: it works. Conclusion: OCR does not work correctly with every font, but it is at least surprising, that it matches correctly, but does not report the correct text. But from outside, we cannot decide, wether this might be weird but consistent internally ;-)

Idea: When matching text, the match should report a similarity based on a string compare search-text vs. text-recognized (e.g percentage of total number of characters to characters equal). So a score of 1.00 is only possible if the text in the match-region is exactly the text we searched for.

I have noticed inconsistency in the Sikuli X text recognition and matching, especially when text are integers. Below are the steps to reproduce the problem scenario:
Step1: In a web browser open the 'TextOnTheScreen.png' using the web links available in my previous post.
Step2: Provide the region coordinates of the 'TextOnTheScreen.png' image to sikuli script provided in my previous post. Note! region coordinates must be significantly bigger because you will zoom-in & zoom-out of the visible content on your browser in-order to carry-out multiple tests.
Step3(TestCase1): Now, run the sikuli script with t = t2. You might notice that 7 & 8 are not recognised correctly or it might recognise all the integer as text right or list only few integers? Well, does it depend on how much OCR gets trained?
Step4(TestCase2): Now, back to your browser where the 'TextOnTheScreen.png' image is visible, zoom-out (ctrl&-) and run the sikuli script with t = t2. You should notice variation in the text recognisation results.
Step5(TestCase3): Repeat TestCase2 with different zoom-levels in your browser, you might get varied text recognisation results.
Step6(TestCase4): Now, run the sikuli script with t = t3. You might notice that integer as text are not recognised at all or does it recognise any integer as text?

Hope this provides some test-cases to improve text or integer recognisation. Well, should text and integer recognisation be independent operations, would this reduced complexity for the OCR?

I also meet with the same problelms with SIKULI OCR engine,will the next release sikuli version improve it?

@ Gary Lee:
If you add a comment to a question, that is not your own, you should subscribe to the question, to be notified, if someone (like me ;-) answers your comment.

So you are lucky now, to get a notice, since I normally check the subscriptions :-)

The answer to your comment:

To give it some pressure ;-)
add your experiences with examples to the related bug and add yourself as affected in the top.

@RaiMan, this broke in 1.1.0? I've had issues with finding text on screen,

m = r.exists("534438",0)

because it's looking for 534438.png, not the text. I built with Tesseract, and have preferences set to do OCR, but still nothing. (Mavericks, Java 7, Sikuli 1.1.0). Bug?

You can try this free online ocr to convert image to text http://www.online-code.net/ocr.html.

In case anyone is curious about this: I am using the SNAPSHOT build from 2019-03-20. This feature seems like it is looking for an image file all the time - I can't seem to get it to recognize a string.

I use this in my script: r.find("9") #just looking for the number 9 in the region 'r'

The script always stops at this line saying "find: ImageMissing: 9"

Unless there is some other way to designate a 'string' rather than an image file path. I tried double quotes, single quotes, and no quotes - all the same result

@Ryan
Sorry for the inconveniences.
Beginning with 1.1.4 you have to use the specific ...text() methods:
see:
https://sikulix-2014.readthedocs.io/en/latest/news.html#revision-of-the-text-find-api

I'm still having issues with text recognition using java.
example: sikulix reads 07/01/2019 as |//||'|/') can you please advise?

Region match = allScreens.findText("Sometext");
allScreens.click(match);
String textRight = match.right(100).text();

it is finding sometext but not reading the text to its right correctly.

hi @RaiMan , I'm using 1.1.4-SNAPSHOT but still text recognition doesn't seem to work. Could you please advise? are there specific settings for Text recognition?