SikuliX

[2.0.4] findText("string") not working --- try with findWord, findWords, findLine, findLines

Asked by Shane Paes

**************************** current state
findText() and find("some text") are supported by the same implementation, which currently does not work well.
This is a bug and will be fixed in 2.0.5

Until then try to solve your problem until with findWord, findWords, findLine and findLines features

**********************************************************

This is my code

        Settings.OcrTextRead = true;
        Settings.OcrTextSearch = true;

        Region RegionShapeMarkup = new Region(4,4,269,165);
        String TextBoxLabelName= "Shape";
        Match TextBoxLabelNameFound = RegionShapeMarkup.find(TextBoxLabelName);
        if (TextBoxLabelNameFound != null)
        {
            RegionShapeMarkup.click (TextBoxLabelNameFound.offset(0, -30));
        }

the script fails with error
FindFailed: Shape.png: (0x0) in R[4,4 269x165]@S(0)

even though searching for text, script fails with error shape.png not found.

Test environment:
Windows 10 64bit, JAVA11

Any help on this issue will be appreciated.

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
RaiMan
Solved:
Last query:
Last reply:
Revision history for this message
masuo (masuo-ohara) said : 		#1

image find API was revised in version 2.0.2
There is an answer for your question.

https://sikulix-2014.readthedocs.io/en/latest/news.html#revision-of-the-image-find-api
The old behavior of find(someText) is now switched off in the standard, but can be switched on by intention (Settings.SwitchToText). But if switched on, the ImageMissing feature is not available. In doubt it is recommended, to change the function calls to their text equivalents, if text search is the intention.

https://sikulix-2014.readthedocs.io/en/latest/news.html#revision-of-the-text-find-api

Revision history for this message
Shane Paes (spaes) said : 		#2

Basically our scenario is, we trying to find a text (Shape) within a region, it does not find the text.
We need to find text itself and not a image. so any help on OCR or text recognition will be helpful

tried the above suggestion, still it does not work.

tried some more optionsv (Tessaract) as well still no luck.
        Settings.OcrDataPath = "C:\\Users\\vizqa\\AppData\\Roaming\\Sikulix\\SikulixTesseract\\tessdata";
        Settings.OcrTextRead = true;
        Settings.OcrTextSearch = true;
        Settings.SwitchToText = true;
        TextRecognizer.start();
       log.info("Clicking on Box Markup");

        Region RegionShapeMarkup = new Region(4,4,269,165);
        RegionShapeMarkup.highlight(10);
        String TextBoxLabelName= "Shape";
        Match TextBoxLabelNameFound = RegionShapeMarkup.exists(TextBoxLabelName, 10);
         if (TextBoxLabelNameFound != null)
        {
            RegionShapeMarkup.click(TextBoxLabelNameFound.offset(0, -30));
        }

Can you please suggest any other thing that we can try? it will be really helpful.

Revision history for this message
Shane Paes (spaes) said : 		#3

We tried to installed sikulixapi latest version as well but it doesnt install anything. give the below message in cmd

java -jar sikulixapi-2.0.4.jar
SikuliX API: nothing to do

Revision history for this message
RaiMan (raimund-hocke) said : 		#4

sikulixapi-2.0.4.jar can only be used as a library/dependency in Java/Java-based projects.

It can run Jython scripts from command line using the option -r as documented for the idea jar.

as mentioned by Masco above:
try with findText() instead of exists, then nothing else has to be done before.

... meaning all this is not needed:
        Settings.OcrDataPath = "C:\\Users\\vizqa\\AppData\\Roaming\\Sikulix\\SikulixTesseract\\tessdata";
        Settings.OcrTextRead = true;
        Settings.OcrTextSearch = true;
        Settings.SwitchToText = true;
        TextRecognizer.start();

If you use the sikulixide....jar, then you can just test simple cases, before transcribing them to Java code.

Revision history for this message
Shane Paes (spaes) said : 		#5

We had already tried the findText() but it didnt work.

is there any other way we can debug this? Is it possible to have a call and show you the issue?

Revision history for this message
masuo (masuo-ohara) said : 		#6

Try simple code.
[for example]
https://drive.google.com/file/d/1RIB7yXdJw8aA4Ui_i0kURxYIgR-VNef9/view?usp=sharing

Revision history for this message
Shane Paes (spaes) said : 		#7

I tired the same with IDE on my machine
m = findText("Launchpad")
m.highlight(3)

got this error

javax.script.ScriptException: ReferenceError: "findText" is not defined in nashorn:mozilla_compat.js at line number 69

Revision history for this message
masuo (masuo-ohara) said : 		#8

＠Shane
You are using JavaScript?
My example is coded by Python.
Try again by Python.

Revision history for this message
Shane Paes (spaes) said : 		#9

@masuo i tried with Python and it worked. How can i confirm that it works for Java?

Revision history for this message
masuo (masuo-ohara) said : 		#10

Please wait @RaiMan' s answer.

Revision history for this message
Shane Paes (spaes) said : 		#11

sure lets wait for @RaiMan's reply.

Revision history for this message
Best RaiMan (raimund-hocke) said : 		#12

I am sorry for the delay - should have gone deeper into the SikuliX implementation from the beginning.
findText() or find("some text") are simply based on a "stupid" implementation (my bad ;-).
I will improve it with the 2.0.5.

Until then try to solve your problem with findWord, findWords, findLine and findLines features.

Here a Java sample:

      Region region = new Region(0, 0, 300, 300);
      try {
        String fImg = new File("testImage/image").getCanonicalPath();
        Match match = region.find(fImg);
        List<Match> lines = match.findLines();
        for (Match m : lines) {
          System.out.println(m.getText());
        }
        match.findLine("API").highlight(2);
        List<Match> words = match.findWords();
        for (Match m : words) {
          System.out.println(m.getText());
        }
        match.findWord("API").highlight(2);
      } catch (Exception e) {
        System.out.println("Error" + e.getMessage());
      }

a "line" in this sense is a row of text in the image, that is seen by Tesseract as a line (same background color, same base line, ...).
a "word" is some text of characters surrounded by non-text (like white space).

Revision history for this message
Shane Paes (spaes) said : 		#13

ok Thanks RaiMan, will give this a try and let you know my observations.

Revision history for this message
Shane Paes (spaes) said : 		#14

Hi RaiMan,
I have tried the above code. It looks promising.
I will continue with more tests.

In the meanwhile for the fix in 2.0.5 release how are you planning to track the findText() or find("some text") fix?

Do you plan to create a bug for it?

Revision history for this message
Shane Paes (spaes) said : 		#15

Hi @RaiMan
Thanks for the above code. it fixed my problem...

I will create this as a bug for findText() or find("some text")

Revision history for this message
Shane Paes (spaes) said : 		#16

Hi @RaiMan
Thanks for the above code. it fixed my problem...

I will create this as a bug for findText() or find("some text")

Revision history for this message
Shane Paes (spaes) said : 		#17

Thanks RaiMan, that solved my question.

Revision history for this message
Chetan (cshamdas) said : 		#18

Hi
is this bug fixed and avialable in 2.0.5?