Mac: Retina: using SikuliX with Java programming

Asked by Robert on 2018-03-22

I am using Sikuli as Java library. I am making a screenshot, and giving path to screenshot, but mous clicks on random places.

Why can it be,

Also can you write an example of path in Mac.

Question information

Language:
English Edit question
Status:
Answered
For:
Sikuli Edit question
Assignee:
No assignee Edit question
Last query:
2018-03-23
Last reply:
2018-03-25
RaiMan (raimund-hocke) said : #1

what Mac?
what Java?
what SikuliX?

Robert (robus27) said : #2

Macbook pro. OS 10.13. 2 version.
Java: 1.8.0 _16-1-b12
SikuliX 1.1.0

On Thu, Mar 22, 2018 at 6:08 PM, RaiMan <
<email address hidden>> wrote:

> Your question #667091 on Sikuli changed:
> https://answers.launchpad.net/sikuli/+question/667091
>
> Status: Open => Needs information
>
> RaiMan requested more information:
> what Mac?
> what Java?
> what SikuliX?
>
> --
> To answer this request for more information, you can either reply to
> this email or enter your reply at the following page:
> https://answers.launchpad.net/sikuli/+question/667091
>
> You received this question notification because you asked the question.
>

RaiMan (raimund-hocke) said : #3

please upgrade to 1.1.2

I guess the Java version is 1.8.0 _161-b12 ???

What exactly are you doing?
paste your code snippet.

Robert (robus27) said : #4

Screen screen = new Screen();
Pattern p = new
Pattern("/Users/goldgames/Documents/SikuliX/ScreenShots/aaa.png");

screen.wait(p,2).click();

This is my code, but now it doesn't work at all.
It shows this error.

Exception in thread "main" FindFailed:
/Users/goldgames/Documents/SikuliX/ScreenShots/aaa.png: (244x172) in
S(0)[0,0 1680x1050] E:Y, T:3.0
  Line 2759, in file Region.java

On Thu, Mar 22, 2018 at 7:32 PM, RaiMan <
<email address hidden>> wrote:

> Your question #667091 on Sikuli changed:
> https://answers.launchpad.net/sikuli/+question/667091
>
> Status: Open => Answered
>
> RaiMan proposed the following answer:
> please upgrade to 1.1.2
>
> I guess the Java version is 1.8.0 _161-b12 ???
>
> What exactly are you doing?
> paste your code snippet.
>
> --
> If this answers your question, please go to the following page to let us
> know that it is solved:
> https://answers.launchpad.net/sikuli/+question/667091/+confirm?answer_id=2
>
> If you still need help, you can reply to this email or go to the
> following page to enter your feedback:
> https://answers.launchpad.net/sikuli/+question/667091
>
> You received this question notification because you asked the question.
>

RaiMan (raimund-hocke) said : #5

FindFailed
means it does not find the image on the screen at the moment it runs.

Please get familiar with the concepts and features of SikuliX:
http://sikulix-2014.readthedocs.org/en/latest/index.html

Robert (robus27) said : #6

The thing is that the image is there.
And as i said, even if there was not any error, it was just clicking on
random place.

Do you think it has been fixed in 1.1.2 build?

On Fri, Mar 23, 2018 at 11:52 AM, RaiMan <
<email address hidden>> wrote:

> Your question #667091 on Sikuli changed:
> https://answers.launchpad.net/sikuli/+question/667091
>
> Status: Open => Answered
>
> RaiMan proposed the following answer:
> FindFailed
> means it does not find the image on the screen at the moment it runs.
>
> Please get familiar with the concepts and features of SikuliX:
> http://sikulix-2014.readthedocs.org/en/latest/index.html
>
> --
> If this answers your question, please go to the following page to let us
> know that it is solved:
> https://answers.launchpad.net/sikuli/+question/667091/+confirm?answer_id=4
>
> If you still need help, you can reply to this email or go to the
> following page to enter your feedback:
> https://answers.launchpad.net/sikuli/+question/667091
>
> You received this question notification because you asked the question.
>

RaiMan (raimund-hocke) said : #7

1.1.2: you should upgrade in any case (1.1.0 I do not supported any more)

Is it only clicking once in a random place?

If yes, you have to check the quality of your image:
http://sikulix-2014.readthedocs.io/en/latest/basicinfo.html#sikulix-how-does-it-find-images-on-the-screen

BTW: Retina display?
if yes, then you have to shoot your Images with the SikuliX IDE.

When using the Mac capture tool, your images will have double pixels in width and height.
SikuliX with Java 8+ compensates the Retina effect automatically.
Mac shots have to be manually shrinked to the half in widt and height to be useable with SikuliX.

Robert (robus27) said : #8

Also keyDown and keyUp methods doesn't work properly.

On Fri, Mar 23, 2018 at 12:23 PM, RaiMan <
<email address hidden>> wrote:

> Your question #667091 on Sikuli changed:
> https://answers.launchpad.net/sikuli/+question/667091
>
> Status: Open => Answered
>
> RaiMan proposed the following answer:
> 1.1.2: you should upgrade in any case (1.1.0 I do not supported any
> more)
>
> Is it only clicking once in a random place?
>
> If yes, you have to check the quality of your image:
> http://sikulix-2014.readthedocs.io/en/latest/basicinfo.html#sikulix-how-
> does-it-find-images-on-the-screen
>
> BTW: Retina display?
> if yes, then you have to shoot your Images with the SikuliX IDE.
>
> When using the Mac capture tool, your images will have double pixels in
> width and height.
> SikuliX with Java 8+ compensates the Retina effect automatically.
> Mac shots have to be manually shrinked to the half in widt and height to
> be useable with SikuliX.
>
> --
> If this answers your question, please go to the following page to let us
> know that it is solved:
> https://answers.launchpad.net/sikuli/+question/667091/+confirm?answer_id=6
>
> If you still need help, you can reply to this email or go to the
> following page to enter your feedback:
> https://answers.launchpad.net/sikuli/+question/667091
>
> You received this question notification because you asked the question.
>

Robert (robusp) said : #9

I will try to make screen shots with SikuliX IDE.

RaiMan (raimund-hocke) said : #10

--- Also keyDown and keyUp methods doesn't work properly.
You will not get any help with such unspecific statements.

We have to ask for more specific information, which is an unnecessary step and time consuming.

So:
what are you doing? (+ code examples)
what are the symptoms?

RaiMan (raimund-hocke) said : #11

... and you did not answer my questions from comment #7

Is it only clicking once in a random place?

BTW: Retina display?

This behavior does not motivate to help you.

RaiMan (raimund-hocke) said : #12

... and screenshots of the IDE do not help and cannot be attached here anyways.

just paste your code snippet here, so someone who decides to try to help you can paste it into its own IDE and check.

Robert (robus27) said : #13

Yes it is just clicking once and finishes.

I will copy my code again.
Is there everything correct in how the path is written?

On Fri, Mar 23, 2018 at 1:26 PM, RaiMan <
<email address hidden>> wrote:

> Your question #667091 on Sikuli changed:
> https://answers.launchpad.net/sikuli/+question/667091
>
> RaiMan proposed the following answer:
> ... and screenshots of the IDE do not help and cannot be attached here
> anyways.
>
> just paste your code snippet here, so someone who decides to try to help
> you can paste it into its own IDE and check.
>
> --
> If this answers your question, please go to the following page to let us
> know that it is solved:
> https://answers.launchpad.net/sikuli/+question/667091/+
> confirm?answer_id=11
>
> If you still need help, you can reply to this email or go to the
> following page to enter your feedback:
> https://answers.launchpad.net/sikuli/+question/667091
>
> You received this question notification because you asked the question.
>

Robert (robus27) said : #14

Yes, my display is retina

On Fri, Mar 23, 2018 at 1:34 PM, Robert P <email address hidden> wrote:

> Yes it is just clicking once and finishes.
>
> I will copy my code again.
> Is there everything correct in how the path is written?
>
> On Fri, Mar 23, 2018 at 1:26 PM, RaiMan <question667091@answers.
> launchpad.net> wrote:
>
>> Your question #667091 on Sikuli changed:
>> https://answers.launchpad.net/sikuli/+question/667091
>>
>> RaiMan proposed the following answer:
>> ... and screenshots of the IDE do not help and cannot be attached here
>> anyways.
>>
>> just paste your code snippet here, so someone who decides to try to help
>> you can paste it into its own IDE and check.
>>
>> --
>> If this answers your question, please go to the following page to let us
>> know that it is solved:
>> https://answers.launchpad.net/sikuli/+question/667091/+confi
>> rm?answer_id=11
>>
>> If you still need help, you can reply to this email or go to the
>> following page to enter your feedback:
>> https://answers.launchpad.net/sikuli/+question/667091
>>
>> You received this question notification because you asked the question.
>>
>
>

Robert (robusp) said : #15

import org.sikuli.script.*;

import java.awt.event.KeyEvent;

public class Main {
    public static void main(String[] args) throws InterruptedException, FindFailed {

        Screen screen = new Screen();
        Pattern p = new Pattern("/Users/goldgames/Documents/SikuliX/ScreenShots/aaa.png");

        screen.wait(p,2).click();

    }
}

RaiMan (raimund-hocke) said : #16

--- Retina
upgrade to 1.1.2 and use the SikuliX IDE to shoot your images

--- handling of the images:
- save a script as "/Users/goldgames/Documents/SikuliX/ScreenShots.sikuli"
- shoot your images using a naming convention
- take care, that the script is saved, before trying to use the images in Java

in Java add once:
ImagePath.add("/Users/goldgames/Documents/SikuliX/ScreenShots.sikuli")

and reference your images this way:
Pattern p = new Pattern("aaa"); // for .png the ending can be omitted
BTW: Pattern is only needed if you want to set a similarity or a targetOffset

so for a simple image:
screen.wait("aaa",2).click()

you might want to handle FindFailed situations with try{}catch(){}

So if it clicks something once, then it has found something similar to the given image.
look at the match:

Match m = screen.wait("aaa",2);
m.click() // just put a debug breakpoint here

--- import java.awt.event.KeyEvent;
... does not make sense in a Java program using SikuliX features

Robert (robusp) said : #18

handling of the images. YES

I am just wondering if i will add imagePath, then to click on it i will be have to just to use name of the image, wont i>

Robert (robusp) said : #19

But when you surround something with try{}catch{}, it might make code work not properly, so isnt it better just to add an exception?

Robert (robusp) said : #20

Also i am unable to check everything and update my SikuliX right now, so I would like you not to close the question and let it opened, so i will be able to report if there will be other issues.

RaiMan (raimund-hocke) said : #21

In a SikuliX workflow having a FindFailed situation either means, that you have done something wrong in preparing your workflow and/or images or that something has happened, that you have foreseen and now be able to recover somehow from the situation.

In the first case, when catching the exception, you might implement some logging and decide how to proceed.
The latter case is obvious.

So, yes it is your decision ;-)

alternatively:

Match m = screen.exists("aaa",2);
if (m == null) {
   // not found - do something
} else {
   m.click() // just put a debug breakpoint here
}

this will never crash.

So it might be, that you have to learn more about exception handling.

RaiMan (raimund-hocke) said : #22

--- I am just wondering if i will add imagePath, then to click on it i will be have to just to use name of the image, wont i>

Using ImagePath.add is strongly recommended in Java code: only in one place you define the location of your image set (DRY principle)

If you do not use it, you have to always give the full path with every mentioned image, which might be weird, if you decide to change your concept.

... but yes, each one has to find its own way ;-)

1.1.0: no problem at this level of usage.
... but I recommend to switch to 1.1.2 as soon as possible.

BTW: what Java IDE are you using?

Robert (robusp) said : #23

I'm using IntelliJ IDE.

So ok, look, for example there is an image i want to use. Do I have to do this?

imagePath.add("Image path/");
Match m = screen.exists("aaa",2);
m.click;

I mean once i added imagfe path with the name of it in the end of path, can i always use the name , and it will be able to find the image always?

RaiMan (raimund-hocke) said : #24

go back to comment #16, read again and try to understand the whole concept.

Add this sample to some test Java, step through it using debug and watch the inspections

... and please read the docs at least once (http://sikulix-2014.readthedocs.io/en/latest/index.html)
... and get comfortable with the JavaDocs and the class/function defs lookup in IntelliJ IDEA (I know, what I am talking about, since I use it too ;-)

Robert (robusp) said : #25

Ok. Thank you very much. I will try to do as you said.
thanks a lot one more time.

Robert (robusp) said : #26

One more question pls.

How can i change the location where screen shots are being saved?
I mean those which have been made by SikuliX IDE

RaiMan (raimund-hocke) said : #27

Currently not a feature in the IDE.

But a solution is mentioned in comment#16

Robert (robusp) said : #28

OK, Great, thank you RaiMan

Can you help with this problem?

Provide an answer of your own, or ask Robert for more information if necessary.

To post a message you must log in.