SikuliX

can VNC connect change SCREEN default ?

Asked by Laurent IZAC

Hi,

I am connecting to VNC and after that I need to prefix all my commands with the screen object returned by the connect call.
Is there a way to set/change the default SCREEN so that I can omit the screen prefix in front of each command ?

What I have:

vs = VNCScreen.start(...)

vs.click(vs.wait("xxx.png",1))
vs.type("xxx")

What I would like :

SCREEN = VNCScreen.start(...)

click(wait("xxx.png",1))
type("xxx")

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
RaiMan
Solved:
Last query:
Last reply:
Revision history for this message
Best RaiMan (raimund-hocke) said : 		#1

You might try with

use(VNCScreen.start(...))

click(wait("xxx.png",1))
type("xxx")

...

use() # reset to primary local screen

... but never tried it with a VNCScreen object - until now only with a second/third local monitor.

Revision history for this message
Laurent IZAC (laurent-izac) said (last edit ): 		#2
Revision history for this message
Laurent IZAC (laurent-izac) said (last edit ): 		#3

Well, it works in the end, but I have to mak sure I use "use(...)" in my top level script/function.
I wanted to embed it in the details of the connection function, but I guess it got applied to a local scope only...

Thanks RaiMan for this very useful tip.

Note: using VNC has the advantage for me that I can open and close sessions as part of my script, leaving no open session to security concerns. It also works when no other remote desktop connection is open (no more black screen). I can have my script started from a Windows service without problem (as it does not interact with a local Windows session, but can interact with a local VNC server).
=> Please do not drop VNC support in your forthcoming releases...

Others:
I noticed that capturing images from TigerVNC client gives best results (as it is the same library used internally by SikuliX).
Also, starting VNC server with other than 24 bit image depth leads to Class cast exceptions when dealing with images.

Revision history for this message
Laurent IZAC (laurent-izac) said : 		#4

Just found an issue with hot keys though.
For example, "type('N', Key.CTRL+Key.SHIFT)" does not work (supposed to open a new Incognito window in Chrome), while it works if I prefix the commands with the VNC Screen object, like : "vs.type('N', Key.CTRL+Key.SHIFT)". Other tested key combinations work just fine, when using only one key modifer
So I suspect there is some Key mapping inconsistency somewhere, at least when using double key modifiers (as witnessed already with CTRL+ALT+DEL)...

Revision history for this message
RaiMan (raimund-hocke) said (last edit ): 		#5

--- "type('N', Key.CTRL+Key.SHIFT)" does not work

I suppose it is used after having issued a use(<vnc-screen>) before.

BTW: type('N', Key.CTRL+Key.SHIFT) is problematic, since the uppercase N already creates a shift-modifier.
So better is: type('n', Key.CTRL+Key.SHIFT).

... but should not have an affect in the mentioned case.

--- So I suspect there is some Key mapping inconsistency somewhere ...
I do not think so.
The cause might be, that in the redirected case (undotted type()) the handling of the parameters of type() is somewhat different.

So for now you have to live with that inconsistency.

I have opened a discussion issue on Github to track things regarding VNC support:
https://github.com/RaiMan/SikuliX1/issues/505