SikuliX

Drawing on the map doesnt work in Windows 7

Asked by sabitha

Scenario is to draw on the map to capture the link id's. For this we are using selenium, JAva with Sikuli Methods.

Issues:
1. In windows 10 it works perfectly. But in windows 7 it doesn't work and fails with below error
2. i gave the path to capture the screenshot and save it in path given below, but it saves it in root directory. Why?
screen.saveScreenCapture(System.getProperty("user.dir")+"\\captured_image");

PSB my code:

try
  {
   WebElement ele = driver.findElement(By.xpath("//*[@id='Map_Div_layers']"));

   int x = ele.getLocation().getX();
   int y= ele.getLocation().getY();

   System.out.println("X: "+x);
   System.out.println("Y: "+y);

   Screen screen = new Screen();

   screen.setLocation(new Location(x,y));

   ((JavascriptExecutor)driver).executeScript("scroll(0,200)");

   System.out.println("Folder to save: "+System.getProperty("user.dir")+"\\captured_image");

   screen.saveScreenCapture(System.getProperty("user.dir")+"\\captured_image");

   //String image=screen.getLastScreenImageFile();

   //System.out.println("Image after screen capture:"+image);

   Pattern p = new Pattern(screen.getLastScreenImage());

   System.out.println("Image in pattern:"+p.getImage());

   driver.findElement(By.xpath("//*[@uib-tooltip='Select Link']")).click();

   System.out.println("Clicked on Select Link");

   ((JavascriptExecutor)driver).executeScript("scroll(0,-80)");

   System.out.println("scrolled up");

   //Match q =screen.find(p);

   //System.out.println("Find and get Image: "+q.getImage());

   //Location mloc=q.getCenter();
   //screen.click(mloc);

   screen.click(p);

   Location mloc = Env.getMouseLocation();

   //screen.doubleClick(p);

   screen.mouseMove(mloc.offset(-260,50));
   screen.click(mloc.offset(-260,50));

   screen.mouseMove(Env.getMouseLocation().offset(40,40));
   screen.click(Env.getMouseLocation().offset(40,40));

   screen.mouseMove(Env.getMouseLocation().offset(20,20));
   screen.click(Env.getMouseLocation().offset(20,20));

   screen.mouseMove(Env.getMouseLocation().offset(0,0));
   screen.click(Env.getMouseLocation().offset(0,0));

   screen.doubleClick(Env.getMouseLocation().offset(0,0));
  }

Below is the log:
Started InternetExplorerDriver server (32-bit)
3.9.0.0
Listening on port 37055
Only local connections are allowed
Sep 26, 2018 10:43:33 AM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
Inside the is_login_page_open method
Checking if login button displayed
Login button displayed: true
Entering username
Entering password
clicking submit button
clicked submit button
Checking Home Page displayed or not
Title: ERP2
X: 798
Y: 316
Sep 26, 2018 10:44:51 AM org.bridj.BridJ log
INFO: Registering type org.sikuli.util.SysJNA$WinKernel32
Sep 26, 2018 10:44:51 AM org.bridj.BridJ log
INFO: Registering type org.bridj.TimeT
Sep 26, 2018 10:44:51 AM org.bridj.BridJ log
INFO: Registering type org.bridj.TimeT$timeval_customizer
Sep 26, 2018 10:44:51 AM org.bridj.BridJ log
INFO: Registering type org.bridj.StructIO$DefaultCustomizer
Sep 26, 2018 10:44:51 AM org.bridj.BridJ log
INFO: Registering type org.bridj.TimeT$timeval
Sep 26, 2018 10:44:51 AM org.bridj.BridJ log
INFO: Registering type org.bridj.StructObject
Sep 26, 2018 10:44:51 AM org.bridj.BridJ log
INFO: Registering type org.bridj.NativeObject
Sep 26, 2018 10:44:51 AM org.bridj.BridJ log
INFO: Registering type org.bridj.AbstractIntegral
Sep 26, 2018 10:44:51 AM org.bridj.BridJ log
INFO: Registering type java.lang.Number
Image:D:\ERP2\Trunk\Selenium\1537929900313.png
Clicked on Select Link
scrolled up
FindFailed: can not find P(1537929900313.png) S: 0.7 in S(0)[0,0 1366x768]
  Line 2189, in file Region.java

        at org.sikuli.script.Region.handleFindFailedShowDialog(Region.java:2189)
        at org.sikuli.script.Region.handleFindFailed(Region.java:2134)
        at org.sikuli.script.Region.wait(Region.java:2546)
        at org.sikuli.script.Region.find(Region.java:2205)
        at serenityDemo.pages.CreateDistanceBasedLocationPage.draw_on_map(CreateDistanceBasedLocationPage.java:79)
        at serenityDemo.steps.serenity.IntranetSanitySteps.verify_user_clicked_on_the_map_to_draw(IntranetSanitySteps.java:230)
        at serenityDemo.steps.serenity.IntranetSanitySteps$$EnhancerByCGLIB$$1225fa0e.CGLIB$verify_user_clicked_on_the_map_to_draw$17(<generated>)
        at serenityDemo.steps.serenity.IntranetSanitySteps$$EnhancerByCGLIB$$1225fa0e$$FastClassByCGLIB$$cf699c5c.invoke(<generated>)
        at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
        at net.thucydides.core.steps.StepInterceptor.invokeMethod(StepInterceptor.java:426)
        at net.thucydides.core.steps.StepInterceptor.executeTestStepMethod(StepInterceptor.java:411)
        at net.thucydides.core.steps.StepInterceptor.runTestStep(StepInterceptor.java:386)
        at net.thucydides.core.steps.StepInterceptor.testStepResult(StepInterceptor.java:134)
        at net.thucydides.core.steps.StepInterceptor.intercept(StepInterceptor.java:63)
        at serenityDemo.steps.serenity.IntranetSanitySteps$$EnhancerByCGLIB$$1225fa0e.verify_user_clicked_on_the_map_to_draw(<generated>)
        at serenityDemo.steps.IntranetStepDefinition.thenUserClickOnTheMapToDraw(IntranetStepDefinition.java:136)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at cucumber.runtime.Utils$1.call(Utils.java:32)
        at cucumber.runtime.Timeout.timeout(Timeout.java:16)
        at cucumber.runtime.Utils.invoke(Utils.java:26)
        at cucumber.runtime.java.JavaStepDefinition.execute(JavaStepDefinition.java:37)
        at cucumber.runtime.StepDefinitionMatch.runStep(StepDefinitionMatch.java:40)
        at cucumber.api.TestStep.executeStep(TestStep.java:102)
        at cucumber.api.TestStep.run(TestStep.java:83)
        at cucumber.api.TestCase.run(TestCase.java:58)
        at cucumber.runner.Runner.runPickle(Runner.java:80)
        at cucumber.runtime.junit.PickleRunners$NoStepDescriptions.run(PickleRunners.java:140)
        at cucumber.runtime.junit.FeatureRunner.runChild(FeatureRunner.java:68)
        at cucumber.runtime.junit.FeatureRunner.runChild(FeatureRunner.java:23)
        at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
        at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
        at cucumber.runtime.junit.FeatureRunner.run(FeatureRunner.java:73)
        at cucumber.api.junit.Cucumber.runChild(Cucumber.java:99)
        at cucumber.api.junit.Cucumber.runChild(Cucumber.java:41)
        at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
        at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
        at cucumber.api.junit.Cucumber$1.evaluate(Cucumber.java:108)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
        at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:367)
        at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:274)
        at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
        at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:161)
        at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:290)
        at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:242)
        at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121)
Map not loaded or links not extracted[Ljava.lang.StackTraceElement;@5546e754
Extracting rows
row Size of links table: 1
[main] ERROR net.serenitybdd.core.Serenity -

The two windows clients has the same monitor and the same resolution ratio.

Dose anyone else has encountered the issue?
Any help is appreciated.

Thx.
Sabitha

Question information

Language:
English Edit question
Status:
Answered
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:

This question was originally filed as bug #1794462.

Revision history for this message
RaiMan (raimund-hocke) said : 		#1

should be a question - surely not a bug

Revision history for this message
sabitha (sabithabogadula) said : 		#2

Hi RaiMan

do you have any update on this?

Thanks
Sabitha

Revision history for this message
RaiMan (raimund-hocke) said : 		#3

--- screen.saveScreenCapture()
... captures the screen content and saves it to the given path as a timestamped PNG file
... the BufferedImage (ScreenImage) from the capture is stored in-memory with the screen object
... it returns the absolute path to the image file

Please check wether this happens.

--- screen.getLastScreenImage()
returns the BufferedImage (ScreenImage), that is stored with the screen object at the time of the last capture

in your case the content that is used with the Pattern() should be the same as that with the screen.saveScreenCapture before, since no new capture could have changed the stored ScreenImage.

Then you do a click with Selenium, which surely changes the screen content.

with the following screen.click(p) you search a former screen content in a current screen content, that has changed and is not found of course.

Even if it would have been found, all this would have benn the same as simply clicking the center of the screen.

Since I have no idea, what you want to achieve, I can only say: revise your workflow

Revision history for this message
sabitha (sabithabogadula) said : 		#4

Hi RaiMan

My motto is explained below

                        WebElement ele = driver.findElement(By.xpath("//*[@id='Map_Div_layers']")); // I am checking whether this element exist or not and taking the x and y coordinates

   int x = ele.getLocation().getX();
   int y= ele.getLocation().getY();

   System.out.println("X: "+x);
   System.out.println("Y: "+y);

   Screen screen = new Screen();

   screen.setLocation(new Location(x,y)); //I am passing the x, y coordinates to set the location

   ((JavascriptExecutor)driver).executeScript("scroll(0,200)"); //i am scrolling down because on the map not every place has links, i scrolling to place where it see roads on the map so it can acquire links

   System.out.println("Folder to save: "+System.getProperty("user.dir")+"\\captured_image"); //after scrolling taking the screen capture and asking to save to that location

   screen.saveScreenCapture(System.getProperty("user.dir")+"\\captured_image");

   Pattern p = new Pattern(screen.getLastScreenImage()); //Sending the last captured image to pattern so i can use the methods to draw

   System.out.println("Image in pattern:"+p.getImage());

   driver.findElement(By.xpath("//*[@uib-tooltip='Select Link']")).click(); //before drawing on the map i need to click on select link

   System.out.println("Clicked on Select Link");

   ((JavascriptExecutor)driver).executeScript("scroll(0,-80)"); //Since the select link is very down of the screen , again i have to scroll up to draw on the map so that it can find the pattern

   System.out.println("scrolled up");

   screen.click(p); //I am clicking on that image

   Location mloc = Env.getMouseLocation(); //taking the mouse location

   screen.mouseMove(mloc.offset(-260,50)); //then draw line to that offset (using mouse move)
   screen.click(mloc.offset(-260,50)); //then click again at that location . Repeating down the same.

   screen.mouseMove(Env.getMouseLocation().offset(40,40));
   screen.click(Env.getMouseLocation().offset(40,40));

   screen.mouseMove(Env.getMouseLocation().offset(20,20));
   screen.click(Env.getMouseLocation().offset(20,20));

   screen.mouseMove(Env.getMouseLocation().offset(0,0));
   screen.click(Env.getMouseLocation().offset(0,0));

   screen.doubleClick(Env.getMouseLocation().offset(0,0));
  }

My main question is when it is running perfectly in Windows 10, why it is not running on windows 7.

Thanks

Revision history for this message
sabitha (sabithabogadula) said : 		#5

Hi RaiMan

My motto is explained below

                        WebElement ele =
driver.findElement(By.xpath("//*[@id='Map_Div_layers']")); // I am checking
whether this element exist or not and taking the x and y coordinates
int x = ele.getLocation().getX();
int y= ele.getLocation().getY();
System.out.println("X: "+x);
System.out.println("Y: "+y);
Screen screen = new Screen();
screen.setLocation(new Location(x,y)); //I am passing the x, y coordinates
to set the location
((JavascriptExecutor)driver).executeScript("scroll(0,200)"); //i am
scrolling down because on the map not every place has links, i scrolling to
place where it see roads on the map so it can acquire links
System.out.println("Folder to save:
"+System.getProperty("user.dir")+"\\captured_image"); //after scrolling
taking the screen capture and asking to save to that location
screen.saveScreenCapture(System.getProperty("user.dir")+"\\captured_image");
Pattern p = new Pattern(screen.getLastScreenImage()); //Sending the last
captured image to pattern so i can use the methods to draw
System.out.println("Image in pattern:"+p.getImage());
driver.findElement(By.xpath("//*[@uib-tooltip='Select Link']")).click();
//before drawing on the map i need to click on select link
System.out.println("Clicked on Select Link");
((JavascriptExecutor)driver).executeScript("scroll(0,-80)"); //Since the
select link is very down of the screen , again i have to scroll up to draw
on the map so that it can find the pattern
System.out.println("scrolled up");
screen.click(p); //I am clicking on that image
Location mloc = Env.getMouseLocation(); //taking the mouse location
screen.mouseMove(mloc.offset(-260,50)); //then draw line to that offset
(using mouse move)
screen.click(mloc.offset(-260,50)); //then click again at that location .
Repeating down the same.
screen.mouseMove(Env.getMouseLocation().offset(40,40));
screen.click(Env.getMouseLocation().offset(40,40));
screen.mouseMove(Env.getMouseLocation().offset(20,20));
screen.click(Env.getMouseLocation().offset(20,20));
screen.mouseMove(Env.getMouseLocation().offset(0,0));
screen.click(Env.getMouseLocation().offset(0,0));
screen.doubleClick(Env.getMouseLocation().offset(0,0));
}

My main question is when it is running perfectly in Windows 10, why it is
not running on windows 7.

Thanks
Sabitha

On Thu, Sep 27, 2018 at 8:03 PM RaiMan <email address hidden>
wrote:

> Your question #674435 on Sikuli changed:
> https://answers.launchpad.net/sikuli/+question/674435
>
> Status: Open => Answered
>
> RaiMan proposed the following answer:
> --- screen.saveScreenCapture()
> ... captures the screen content and saves it to the given path as a
> timestamped PNG file
> ... the BufferedImage (ScreenImage) from the capture is stored in-memory
> with the screen object
> ... it returns the absolute path to the image file
>
> Please check wether this happens.
>
> --- screen.getLastScreenImage()
> returns the BufferedImage (ScreenImage), that is stored with the screen
> object at the time of the last capture
>
> in your case the content that is used with the Pattern() should be the
> same as that with the screen.saveScreenCapture before, since no new
> capture could have changed the stored ScreenImage.
>
> Then you do a click with Selenium, which surely changes the screen
> content.
>
> with the following screen.click(p) you search a former screen content in
> a current screen content, that has changed and is not found of course.
>
> Even if it would have been found, all this would have benn the same as
> simply clicking the center of the screen.
>
> Since I have no idea, what you want to achieve, I can only say: revise
> your workflow
>
> --
> If this answers your question, please go to the following page to let us
> know that it is solved:
> https://answers.launchpad.net/sikuli/+question/674435/+confirm?answer_id=2
>
> If you still need help, you can reply to this email or go to the
> following page to enter your feedback:
> https://answers.launchpad.net/sikuli/+question/674435
>
> You received this question notification because you asked the question.
>

Revision history for this message
RaiMan (raimund-hocke) said : 		#6

--- My main question is when it is running perfectly in Windows 10, why it is
not running on windows 7.

I cannot tell you. you have to find out by isolating the offending stuff.

Can you help with this problem?

Provide an answer of your own, or ask sabitha for more information if necessary.

To post a message you must log in.