Mac11.5.2 Mouse not usable (blocked)

Asked by lisong2

Mac11.5.2 Mouse not usable (blocked)

both SikulixIDE and Java invoke gives me this error, how to fix?

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
lisong2
Solved:
Last query:
Last reply:
Revision history for this message
lisong2 (lisong2) said :
#1

SikuliX version 2.0.5

Revision history for this message
RaiMan (raimund-hocke) said :
#2
Revision history for this message
lisong2 (lisong2) said :
#3

tried, not work

Revision history for this message
lisong2 (lisong2) said :
#4

always see the mouse error whenever I start SikuliXIDE from a terminal, from both a popup window and the right side in IDE

Revision history for this message
RaiMan (raimund-hocke) said :
#5

Do clicking and screenshots work?
If yes, just ignore the message.

If not, restart from scratch step for step.

Revision history for this message
lisong2 (lisong2) said :
#6

not working...

I tried this in my Java code:

Screen s = new Screen();

and gives me the mouse error (however it move my mouse to the center of my screen)

Revision history for this message
lisong2 (lisong2) said :
#7

don't know how to post a pic

but I am sure I have 'java, terminal and IntelliJ IDEA' in my Syspref Accessibility and Screen Recording

Revision history for this message
RaiMan (raimund-hocke) said :
#8

as mentioned:
try a click and use the capture button in the IDE.

If that works, just ignore the messages (should only happen once)

Revision history for this message
lisong2 (lisong2) said :
#9

thanks the IDE works now, after I totally start over from the beginning

just one more issue, my idea doesn't work for sikuli

my pom:
            <groupId>com.sikulix</groupId>
            <artifactId>sikulixapi</artifactId>
            <version>2.0.5</version>

my code:
Screen s = new Screen();
String imgpath = "/Users/admin/Desktop/";
s.doubleClick(imgpath+"apple.png");
System.out.println("Done");

and I always get:

[error] Mouse: not useable (blocked)
See: https://github.com/RaiMan/SikuliX1/wiki/Allow-SikuliX-actions-on-macOS
Exception in thread "main" org.sikuli.script.SikuliXception: Mouse.init: Mouse not useable (blocked) - Screenshots might not work either!
 at org.sikuli.script.Screen.initScreens(Screen.java:184)
 at org.sikuli.script.Screen.<init>(Screen.java:198)

can you please help?

Revision history for this message
RaiMan (raimund-hocke) said :
#10

again: does the doubleClick work?

If that works, just ignore the messages (should only happen once)

Revision history for this message
lisong2 (lisong2) said :
#11

thanks the IDE works now, after I totally start over from the beginning

just one more issue, my idea doesn't work for sikuli

my pom:
            <groupId>com.sikulix</groupId>
            <artifactId>sikulixapi</artifactId>
            <version>2.0.5</version>

my code:
Screen s = new Screen();
String imgpath = "/Users/admin/Desktop/";
s.doubleClick(imgpath+"apple.png");
System.out.println("Done");

and I always get:

[error] Mouse: not useable (blocked)
See: https://github.com/RaiMan/SikuliX1/wiki/Allow-SikuliX-actions-on-macOS
Exception in thread "main" org.sikuli.script.SikuliXception: Mouse.init: Mouse not useable (blocked) - Screenshots might not work either!
 at org.sikuli.script.Screen.initScreens(Screen.java:184)
 at org.sikuli.script.Screen.<init>(Screen.java:198)

can you please help?

Revision history for this message
lisong2 (lisong2) said :
#12

no, error was thrown on the 1st line (Screen s = new Screen();)

Revision history for this message
RaiMan (raimund-hocke) said :
#13

just tested and switched off the accessibility usages before:

public class SikulixTest {
  public static void main(String[] args) {
    System.out.println("SikulixTest");
    System.out.println("SikulixTest screen failing");
    Screen screen = null;
    try {
      screen = new Screen();
    } catch (Exception e) {
    }
    System.out.println("SikulixTest screen ok");
    screen = new Screen();
    System.out.println("SikulixTest screen used");
    screen.hover();
  }
}

gives the following output:
SikulixTest
SikulixTest screen failing
[error] Mouse: not useable (blocked)
See: https://github.com/RaiMan/SikuliX1/wiki/Allow-SikuliX-actions-on-macOS
SikulixTest screen ok
SikulixTest screen used
[error] Mouse.move(): Mouse not useable (blocked)

after switching the accessibility rights on again:
SikulixTest
SikulixTest screen failing
SikulixTest screen ok
SikulixTest screen used

and the mouse moves to the center of the screen

Revision history for this message
lisong2 (lisong2) said :
#14

Thanks! mouse issue resolved, but new error comes on the hover() method:
screen.hover(); works,

but
screen.hover(imgpath+"apple.png"); gives me below error:

Exception in thread "main" CvException [org.opencv.core.CvException: cv::Exception: OpenCV(4.3.0) /Users/jason/Projects/openpnp/opencv/opencv/opencv-4.3.0/modules/imgproc/src/templmatch.cpp:838: error: (-213:The function/feature is not implemented) in function 'matchTemplateMask'
]
 at org.opencv.imgproc.Imgproc.matchTemplate_0(Native Method)
 at org.opencv.imgproc.Imgproc.matchTemplate(Imgproc.java:7485)
 at org.sikuli.script.Finder$Finder2.doFindMatch(Finder.java:745)
 at org.sikuli.script.Finder$Finder2.doFindImage(Finder.java:706)
 at org.sikuli.script.Finder$Finder2.doFind(Finder.java:624)
 at org.sikuli.script.Finder$Finder2.find(Finder.java:584)
 at org.sikuli.script.Finder.find(Finder.java:221)
 at org.sikuli.script.Region.runFinder(Region.java:2861)
 at org.sikuli.script.Region.doFind(Region.java:2799)
 at org.sikuli.script.Region.access$200(Region.java:29)
 at org.sikuli.script.Region$RepeatableFind.run(Region.java:3057)
 at org.sikuli.script.Region$Repeatable.repeat(Region.java:3002)
 at org.sikuli.script.Region.wait(Region.java:2200)
 at org.sikuli.script.Region.wait(Region.java:2244)
 at org.sikuli.script.Region.getLocationFromTarget(Region.java:3298)
 at org.sikuli.script.Region.mouseMove(Region.java:4209)
 at org.sikuli.script.Region.hover(Region.java:3864)
 at main.main(main.java:18)

so looks like my opencv is not working, do you have any intel? since sikuliX-IDE totally works for me when mouse hover on a pic, but java not.

Revision history for this message
RaiMan (raimund-hocke) said :
#15

how did you create the image "apple.png"?

Should be captured using the IDE.

And for tests use an image that is not on the top menu bar.

Revision history for this message
lisong2 (lisong2) said :
#16

Thanks! I captured the png via IDE, and it is not on the top menu bar.

click() works in my Java now.

But hover(), mouseMove() still not working (both in sikulix-IDE and Java)

can you please help again ...?

Revision history for this message
RaiMan (raimund-hocke) said :
#17

... But hover(), mouseMove() still not working (both in sikulix-IDE and Java)

please post code sample and error message

Revision history for this message
lisong2 (lisong2) said :
#18

Screen screen = null;
        try {
            screen = new Screen();
        } catch (Exception e) {
        }
        System.out.println("SikulixTest screen ok");
        screen = new Screen();

        System.out.println("SikulixTest screen used");
        String imgpath = "/Users/admin/Desktop/";
        screen.mouseMove(imgpath+"apple.png");
// screen.hover(imgpath+"apple.png");

        System.out.println("Done");

output:
[error] Mouse: not useable (blocked)
See: https://github.com/RaiMan/SikuliX1/wiki/Allow-SikuliX-actions-on-macOS
SikulixTest screen ok
SikulixTest screen used
[error] Mouse.move(): Mouse not useable (blocked)
Done

Process finished with exit code 0

Revision history for this message
lisong2 (lisong2) said :
#19

please ignore the name of the pic, I just didn't change it, the pic is captured by IDE, and it is a screenshot of my code in my Idea IDE
this line (Screen screen = null;)

again, screen.click() works for the same pic

Revision history for this message
RaiMan (raimund-hocke) said :
#20

- hover and mouseMove are the same (hover internally issues mouseMove)

Does the mouse move to the image area? If yes, ignore the message.

I have this from time to time also, when running java stuff from inside IDEA.
It is because the mouse is not moved exactly to the point, but some pixels away.
I am working on that in 2.0.6.

Revision history for this message
lisong2 (lisong2) said :
#21

OK, thanks for this info.

Actually, mouseMove and hover are not very important to me as of now.

I am using 2.0.5 and looking forward to 2.0.6 :)

Revision history for this message
Anuj (anuj-shrivastava07) said :
#22

Hey @lisong2 - how did you resolved

[error] Mouse: not useable (blocked)

I am using Windows 10 and below is my code

  Screen screen = new Screen();
  Pattern password = new Pattern("/Automation/images/1668142350312.png");
  screen.click(password);

I already tried doubleClick and directly passing image in screen.click("/Automation/images/1668142350312.png") but no luck