Defined Region is getting corrupted if I enter it into a While loop

Asked by Sagar Chakraborty

Hi, I am new to sikuli, so request you all to spoon feed. I am using Sikuli with Eclipse in Java. I am trying to click on a specific region from a while loop. Please find below my problem in detail:-
I have a grid which contains 15 columns and 2 rows. The first row contains the Heading for each of the column and the 2nd row contains corresponding values (which are not fixed; so I can't click on them directly). I have to click on the value field for a specific Column heading and I am doing this by taking the region of the value field by passing the image of the Column heading. I have different functions for getting region of image, clicking on a region by the reference of another image, even for clicking an image.
Now; here's the code I am using to get the region of a specific image. Using this code, I am getting the region of the heading, and the corresponding value.

public static void GetRegionOfImage(String Directory where the Image is kept for value/Heading) throws Exception {

  try{
  Match computerScreenshot = sikuliObj.find(strImage);
   int xb = computerScreenshot.x;
   int yb = computerScreenshot.y;
   int wb = computerScreenshot.w;
   int hb = computerScreenshot.h;
   System.out.println("The co-ordinate of the image "+strImage+" is: X is"+xb+", Y is "+yb+", W is "+wb+", H is "+hb+" ");
   /*Highlighting the result area*/

  Region rd=new Region(xb,yb,wb,hb);
  rd.highlight(4);
  }catch (Exception e) {
   e.getMessage();
   System.out.println("Some error in getting the region of the Image "+strImage+"- entered in Catch block");
  }
  //return rd;
 }

Now after getting the Co-ordinate for both the heading as well as the value, I am using the below code to get the region of the value field by passing the image of the heading field.

public static Region GetRegion(String Directory where the Image is kept for Heading,int x,int y,int w,int h) throws Exception {

  try{
  Match screenShot = sikuliObj.find(strImage);
   int x1 = screenShot.x;
   int y1 = screenShot.y;
   int w1 = screenShot.w;
   int h1 = screenShot.h;
   System.out.println("X is"+x1+", Y is "+y1+", W is "+w1+", H is "+h1+" ");
   /*Highlighting the result area*/
   @SuppressWarnings("deprecation")
   Region rd=new Region(x1+x,y1+y,w1+w,h1+h);

  return rd;
  }catch (Exception e) {
   e.getMessage();
   System.out.println("Some error in getting the region of the Field corresponding to "+strImage+"- entered in Catch block");
   return null;
   }
 }

Now i am executing the below code from Main() to click on the specified value w.r.t. column heading:-
public static Screen sikuliObj = new Screen();
Region rd10=GetRegion(Directory where the Image is kept for Heading,2,59,-3,-2);
rd10.highlight(4);
sikuliObj.click(rd10);
This is clicking the exact region of value. But the trick is:- only 8 out of 15 headings are viewable when I enter into the page for the 1st time. There is an arrow icon just below the grid. If I click on this Icon, the rest of the 7 Headings will be shown one by one. Now, if I need to click on the value corresponding to the 9th or 10th heading (for example), I cannot click it just after I come into the page. I have to click the Arrow icon once or twice and then only I will be able to click on the value corresponding to the 9th or 10th heading. Now I am trying to do this by using a while loop inside Main(). I have kept everything just the same but somehow, sikuli is not clicking the correct region inside while loop. The region is getting corrupted. It is clicking in a completely differnet region everytime. However; I have checked that, it is identifying the root image (Image of the Heading)correctly.But, it is the region of the value field which is getting corrupted. Please suggest how to do this. I will have to continue clicking on the Arrow until I get the heading which I have sent through the image.

ImgStrtWin1= Location for Image of the arrow Icon;
boolean g=true;
while(g)
   {
     boolean Flag1=IsImagePresent(Directory where the Image is kept for Heading);
     if (Flag1== true) {
      //Settings.MinSimilarity=.9;
      Region rd10=GetRegion(Directory where the Image is kept for Heading,2,59,-3,-2);
      rd10.highlight(4);
      sikuliObj.click(rd10);
      System.out.println("Clicked the image of room type "+Directory where the Image is kept for Heading+" in one go- all good..");
      g = false;

      }
     else {
      ClickImage(ImgStrtWin1);
      System.out.println("It took some loops to click the image of room type "+ImgStrtWin48+" all good..");

     }
   }
I have tried this with try-catch also; but no luck.
The functions ClickImage(Image location) and IsImagePresent(Image location)are here for your reference:-
public static void ClickImage(String Directory where image is kept) throws Exception {
   Screen Screen2 = new Screen();
  try {
   if (Screen2.find(strImage) != null) {
    Screen2.click(strImage);
    System.out.println("Clicked the image "+strImage+"- all good..");
   }

  } catch (Exception e) {
   e.getMessage();
   System.out.println("Some error in clicking the image "+strImage+"- entered in Catch block");

  }
 }

public static boolean IsImagePresent(String Directory where image is kept)throws Exception{

     try{
      if(sikuliObj.find(strImage)!=null)
      {

       Region rd=sikuliObj.find(strImage).inside();
       rd.highlight();
       return true;

      }
      else
      {
       return false;
      }
     }
     catch(Exception e)
     {
      e.getMessage();
      return false;
     }

    }

Question information

Language:
English Edit question
Status:
Answered
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said :
#1

--- supposing you are using version 1.0.1 (since you are new to Sikuli)

Some general comments first.

--- This is how I would write these functions:

public static Region GetRegionOfImage(String strImage) {
  try{
  Match computerScreenshot = sikuliObj.find(strImage);
  Debug.user("The co-ordinate of the image %s is %s", strImage, computerScreenshot); // import org.sikuli.basics.Debug;
  // usage like String.format()
  computerScreenshot.highlight(4);
  }catch (Exception e) {
   Debug.user("Some error in getting the region of the Image %s - entered in Catch block\n%s", strImage, e.getMessage());
  }
  return computerScreenshot;
 }

A Match is a Region as well (Match extends Region) and can be used as such.

--- @SuppressWarnings("deprecation")
   Region rd=new Region(x1+x,y1+y,w1+w,h1+h);

This is generally not recommended - deprecations are not there for fun.

on the Java level you should use Region.create()
   Region rd = Region.create(x1+x,y1+y,w1+w,h1+h);

--- the loop:

ImgStrtWin1= Location for Image of the arrow Icon;
while(true)
   {
     if ( IsImagePresent(Directory where the Image is kept for Heading)) {
      //Settings.MinSimilarity=.9;
      Region rd10=GetRegion(Directory where the Image is kept for Heading,2,59,-3,-2);
      rd10.highlight(4);
      sikuliObj.click(rd10);
      System.out.println("Clicked the image of room type "+Directory where the Image is kept for Heading+" in one go- all good..");
      break;
      }
     else {
      ClickImage(ImgStrtWin1);
      System.out.println("It took some loops to click the image of room type "+ImgStrtWin48+" all good..");
      wait(1); // give the GUI some time to scroll
     }
   }

Revision history for this message
RaiMan (raimund-hocke) said :
#2

If you like, you can send me screenshots, so I better understand the GUI

silently use my mail at https://launchpad.net/~raimund-hocke

Revision history for this message
Sagar Chakraborty (sagarc835) said :
#3

Thanks a lot RaiMan for your quick response but sadly; this did not help me. I understood taht, identifying a region can be done in a better and more efficient way (as suggested by you) and I will follow this in my code but; the problem is happening only when I am entering the existing code for identifying the region in the While loop.

Kindly suggest how this can be done. Already shared the screenshots in your mail ID.

Revision history for this message
RaiMan (raimund-hocke) said :
#4

Sent you an example, how I would find my way through a grid GUI like yours

Can you help with this problem?

Provide an answer of your own, or ask Sagar Chakraborty for more information if necessary.

To post a message you must log in.