[1.1.1] click() doesn't work on Windows10 - 1709 with Java 8 latest - Java/Windows HiDPI problem

Asked by landylan

---- info on related bug 1730645 and the test
The test app (plain Java AWT Robot) works for me on:
Windows 10 Version 1709 Build 16299.192 (latest per today 2018-02-05)
Java 8 version 152
Normal monitor setup 1920 x 1200, scaling 100%, no HiDPI

The problem seems to exist with Java 8 not supporting HiDPI.

Using Java 9 on HiDPI systems with a scaling of 100% seems to work (but SikuliX is not tested on Java 9 in all aspects and might have the one or other glitch).
------------------------------------------------------------

Hello,

Thanks for this great project, it helps a lot for many of us.

But there's a big problem for click() since 2017/11/27. It just doesn't work as expected. When we run the one line script below:

click(Location(500,500))

We got the response:

[error] RobotDesktop: checkMousePosition: should be L(500,500)@S(0)[0,0 3840x2160] but after move is L(0,0)@S(0)[0,0 3840x2160] Possible cause in case you did not touch the mouse while script was running: Mouse actions are blocked generally or by the frontmost application. You might try to run the SikuliX stuff as admin.
[log] CLICK on L(500,500)@S(0)[0,0 3840x2160] (570 msec)

We've tried some suggestion in https://answers.launchpad.net/sikuli/+question/251336, but it doesn't work...

It seems related to the win10 update... May we ask if this will be handled and solved? Or is there any trick or hack we can do to let our script work again?

Sorry for bothering you about this. But it will be very helpful for us if this work again...

Question information

Revision history for this message
RaiMan (raimund-hocke) said :
#3

@landylan
Sorry for the inconvenience.

I again tested on my Windows 10 latest build today and cannot confirm any problems. It simply works for me.

Since there are only a few complaints about that mouse move problem, I suppose it is caused by a very special system configuration of the grafics hardware and the related drivers.

Apparently this is related to some glitch between Java and Windows:
https://bugs.openjdk.java.net/browse/JDK-8196030

Revision history for this message
masuo (masuo-ohara) said :
#4

@RaiMan
This is a tentative workaround to make the script to work.
I do not know the perfect solution.

Before Windows10 update, this code worked correct.
Settings.MoveMouseDelay = 0

After Windows10 update, that code don't work correct.
So I tried this codes.
Settings.MoveMouseDelay = 0.1

Although not the same,it worked.

Revision history for this message
landylan (landylan-163) said :
#6

I've tried the test in https://bugs.launchpad.net/sikuli/+bug/1730645 and the error still appears for me.

So it is the problem of Java and Win10?

I then change to another machine(Surface Pro 3), which I restored to factory default a few weeks ago, and I install java, and run the test again... There's still same error.... Even I use the "Settings.MoveMouseDelay = 0.1", it is still the same.

I may try to restore my SP3 to factory default again later, and install pure java to run the test again. If it is the problem of java, will there be any way to get rid of this problem?

Revision history for this message
RaiMan (raimund-hocke) said :
#9

@landylan
Apparently something vital with mouse-move has been changed in Windows (perhaps only together with some HiDPI and/or graphics device setups - I do not know), that affects the use of Java AWT Robot in these situations.

As I understand, there is no possible solution in the near future.

As mentioned: I do not have the problem on my Windows 10 box with a normal graphics setup.
I do not have any experiences with the Surface devices ( .. and sorry, will surely never have ;-).

Revision history for this message
landylan (landylan-163) said :
#11

@RaiMan

Thanks for your information of https://bugs.openjdk.java.net/browse/JDK-8196030 .

It says the problem appear when screen scale is set to 125% or greater. My setting is 150%, so I restore to 100%, but it still doesn't work.

And I notice that there's a comment say AWT does not support HiDPI in JDK8. So I install JDK9 and it work!! IT WORKS!! ^_^

Then I go back to my original PC, and install JDK9. Before I change my screen scale(150%), it still doesn't work, but after I change to 100%, it finally work!!

CONCLUSION:
===============
This problem appears on HiDPI because AWT doesn't supported it. Temperal solution is listed below:
1. Install JDK9
2. Set windows screen scale to 100%

Hope this helps. Thank you all for your kindly help!! ^_^

Revision history for this message
masuo (masuo-ohara) said :
#12

@landylan
Thank you for your information.

Revision history for this message
RaiMan (raimund-hocke) said :
#13

@masuo
ok, understood.
... but this is a completely other problem:
with a delay of 0 it might happen, that after the previous action, the GUI is not yet ready for accepting the click at the move target position.
This was the reason behind, why the original developers 5 years ago implemented the so called "animated mouse move", that for some time (0.5 seconds in the standard) moves the mouse in steps towards the target position (beginning with larger steps getting smaller the nearer the target).

So using a delay of 0 only makes sense in special situations (e.g. in games).

Revision history for this message
RaiMan (raimund-hocke) said :
#14

@masuo
... so I will hide your comments on this thread to not irritate other people.

Revision history for this message
landylan (landylan-163) said :
#15

wait.... the situation is not so stable.... I'll keep trying and report later... thanks for your patient...

Revision history for this message
RaiMan (raimund-hocke) said :
#16

@landylan
no problem ;-)

come back with any additional information.
Thanks for your efforts and cooperation.

Revision history for this message
landylan (landylan-163) said :
#17

Here're some information FYR:

- JDK9 + screenScale:100% + sikulix1.1.2 + reboot ==> PROBLEM SOLVED!

Although I got some unstable situation before I reboot:
    - "java -jar MoveMouseAround.jar loops delay" TEST is OK for most of time (there's still 1 time abnormal during 10+ times TEST).
    - "click(Location(500,500))" one line script TEST is NG for most of time (there're only a few times OK in 10+ times TEST).

But after I reboot my PC, everything's good. No more error for the 2 TEST above.

I've also installed sikulix 1.1.2 and keep 1.1.1 in another directory. 1.1.2 works for me, so I go back to check the 1.1.1 and found that it cannot be launched anymore. According to the message, it seems 1.1.1 is not compatiable with JDK9. I'm not sure about this, but I think it is enough if 1.1.2 works fine for me.

Although click() is ok now, but there're some warning and error message in command window when sikulix launched. I'm not sure if there's any other issue as Raiman said. I'll try to keep using it because it is the only way for me. If there's any other issue, I'll go back here to let you know.

Thank you very much for all your kind help!!

Revision history for this message
RaiMan (raimund-hocke) said :
#18

ok, thanks.

Java 9: 1.1.1 might not work, depending on what features you use.
You might download setup again (now has a fix to accept Java 9) and run a new setup - but no guarantee.

Version 1.1.2 has some fixes for Java 9, but still says that use of Java 9 is experimental.

I will now switch to Java 9 on my Windows 10 system and try to make 1.1.2 Java 9 executable.

Revision history for this message
landylan (landylan-163) said :
#19

information FYR.

Everything's good until now. One problem is that hotkey doesn't work.

[error] RunTimeIDE: *** JAVA9: JintelliType HotkeyHandling not useable - skipped

That means alt+shift+c (exit script) doesn't work now.

Revision history for this message
RaiMan (raimund-hocke) said :
#20

I will try to fix some more Java 9 related problems the next days.

Meanwhile use other means to kill a running script.

Revision history for this message
RaiMan (raimund-hocke) said :
#21

@landylan
The hotkey feature is back ;-)

1.1.2 nightly build 99

Revision history for this message
landylan (landylan-163) said :
#22

WOW! Amazing!! ^_^

I'll try it and report if there's anything need to be report , although everything good is even better. ^_^