SikuliX

Game Testing - Click not always working

Asked by MicMarcil

Hello,

I'm evaluating Sikuli to test games.
The game is built using Unity3D engine, a quite popular game engine.

We have a basic HUD with buttons, opening pop-ups and stuff... but the click is not working sometimes.

Sikuli logs the click, so I assumed it happened. No error message.

The find is working, so it's not a match problem.
I made sure the app has focus before doing the click.
I tried running the game as adminsitrator.
I'm launching the script from the IDE, using Sikuli 1.0.1
When Launching in Slow Motion, the script always target the right place... but the click fail sometimes.
I'm not running my game in fullscreen. It's windowed.

Sometime it works, sometime the click never happens... :(

Is there any known issues when using Sikuli with games ? Especially with the Unity3D engine?
Is there a parameter to adjust/tweak the click speed ? Maybe the click is happening too fast/too slow, sometimes?

Thank you for your feedback.
Michael

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
RaiMan
Solved:
Last query:
Last reply:
Revision history for this message
Test (c4456517) said : 		#1

You can try using some lower level mouse functions to alter the click time.

mouseDown(Button.LEFT)
wait(1)
mouseUp()

This should make the 'click' last slightly longer, if it needs to be more, change the wait timing.

Revision history for this message
Best RaiMan (raimund-hocke) said : 		#2

comment #1 is the right approach, to fix the problem, but usually in these cases
wait(0.3)
is sufficient.

With version 1.0.1+ there is a convenience:
Settings.ClickDelay = 0.3
click(some_image_or_pattern)

or to make it more visible in the code:
Settings.ClickDelay = 0.3; click(some_image_or_pattern)

This just internally inserts a short wait after the mouseDown internally (as in comment #1) and only applies to the next click.

*** The background:
Some app specific GUI's have a short delay after the mouse enters an element before accepting a click. This often comes together with some tooltip-like display of information.
Others need a delay between mouseDown and mouseUp to interpret this a s a click. This depends on implementation.
And there have been situations, where the implementation had some flaws, so depending on cpu usage and parallel processes these delays varied (which should be taken as a bug ;-).

so the general "mouse-flow" for a click can be seen as this:
hover(target)
wait(x)
mouseDown()
wait(y)
mouseUp()

where x and y usually are 0 or in some cases some 10 milliseconds (usually < 300).

For the y value we have the above mentioned convenience with version 1.0.1
hover(target)
wait(x)
Settings.ClickDelay = 0.3; click(some_image_or_pattern)

Revision history for this message
RaiMan (raimund-hocke) said : 		#3

sorry, correction:

For the y value we have the above mentioned convenience with version 1.0.1
hover(target)
wait(x)
Settings.ClickDelay = 0.3; click()

Since click() is a shortcut for click(getLastMatch())

and if we have a restricting region:
some_reg.hover(target)
wait(x)
Settings.ClickDelay = 0.3; some_reg.click()

Revision history for this message
MicMarcil (micmarcil) said : 		#4

The Settings.ClickDelay was the answer to my problem.

Thank you very much ! I've struggled several hours before posting here. Should have done that in the first place :)

I notice the documentation on the website only reference 1.0.0 functions and fields.
Am I wrong ?

I never found the ClickDelay field in the documentation, here:
http://doc.sikuli.org/globals.html

Revision history for this message
MicMarcil (micmarcil) said : 		#5

Thanks RaiMan, that solved my question.

Revision history for this message
RaiMan (raimund-hocke) said : 		#6

yes, sorry, the docs are still behind.

but I promise to get better here also ;-)

thanks for feedback