Reading a dynamic text relative to an image

Asked by Payam Fard

I want to read a dynamically generated text in my desktop application relative to an image. I am using the following code to do so:

Screen objScreen = new Screen();
String dynamicValue = objScreen.wait(imagesPath + "home.PNG").offset(64, 0).text();

Basically, home.PNG is the image that Sikuli is able to find, and I am trying to read the dynamic value based on the offset. It finds and reads the value into dynamicValue but it partially reads it. Would it be possible to expand the area of where text() function is reading in the code above? Or is there a better way to do this in Sikuli?

Question information

Language:
English Edit question
Status:
Answered
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said :
#1
Revision history for this message
Payam Fard (pfard) said :
#2

Thanks for your response. I tried grow(), and it is better, but for some reason, it is reading a strange additional character and removing a character. Basically, with the example below, I am opening up Wordpad and based on the image of "Home" menu on top, I am trying to read "View" menu into the value variable. For some reason, without the addition of grow(), it was reading "Vie" and now with the addition of grow, it is reading "Vie
E" instead of "View".

Not sure why "w" is not being read and not sure where "E" on the second line is coming from. Any thoughts?

[code]
import java.util.Iterator;

import org.sikuli.basics.Debug;
import org.sikuli.basics.Settings;
import org.sikuli.script.*;

public class TestSikuli {

    public static void main(String[] args) throws InterruptedException {
            Screen s = new Screen();
            Debug.setDebugLevel(3);

     Settings setting = new Settings();
   setting.OcrTextSearch = true;
   setting.OcrTextRead = true;

            try{
             App region = App.open("C:\\Program Files\\Windows NT\\Accessories\\wordpad.exe");
             Region appRegion = region.focusedWindow();
             appRegion.highlight(1);

             // get the project folder absolute path
             String workingDir = System.getProperty("user.dir");

             ImagePath.add(TestSikuli.class.getCanonicalName() + "/images");
             String imagesPath = workingDir + "\\images\\";

             System.out.println("images path is " + imagesPath);

             Screen objScreen = new Screen();

             objScreen.type("Hello, Sikuli!");

             Iterator<Match> matchedObjects = objScreen.findAllText("Sikuli");
             Match singleMatch = null;

             if(matchedObjects.hasNext()) {
              System.out.println("------------------- Inside if #1 -----------------------");
              singleMatch = matchedObjects.next();
              System.out.println("------------------- Found object is : " + singleMatch.text());
             }

             Iterator<Match> matchedObjects1 = objScreen.findAllText("Paint");
             Match singleMatch1 = null;

             if(matchedObjects1.hasNext()) {
              System.out.println("------------------- Inside if #2 -----------------------");
              singleMatch1 = matchedObjects1.next();
              System.out.println("------------------- Found object is : " + singleMatch1.text());
             }

             objScreen.wait(imagesPath + "home.PNG").offset(50, 0).grow(25,0).click();

             String value = objScreen.wait(imagesPath + "home.PNG").offset(50, 0).grow(25,0).text();

             System.out.println("------------------- Value is -----------------------" + value);

            }
            catch(FindFailed e){
                    e.printStackTrace();
            }

    }
}
[/code]

Revision history for this message
RaiMan (raimund-hocke) said :
#3

Might be due to the known weaknesses of the text feature.

To check wether the text you want to have can be read correctly, just run this oneliner in the SikuliX IDE:

uprint(selectRegion().text())

this will allow you to select a region containing text and watch the result.

take care, that at best only the area containing the text is selected.

You simply have to live with the results.
May be your experiences help you, to make your script better.

Can you help with this problem?

Provide an answer of your own, or ask Payam Fard for more information if necessary.

To post a message you must log in.