SikuliX

Using Sikuli in multiple RDP connections

Asked by Cedar

Hi there,

First of all, thank you so much for all the work you do. Sikuli is the great tool, and it is really great you support it and move it forward. With Sikuli, you help lots of people to solve issues they face in automation.

My question seems to be a little dummy in some sense, but it is quite important at the same time. It is a dummy because I don't understand in the full extent how Sikuli emulates mouse clicks and keyboard typing, and it is important because we rely on Sikuli heavily.

Prehistory:

So we are using Sikuli a lot in our automated tests. Here is how it looks like: there are some Windows machines (aka dispatchers) where automated tests run. On each dispatcher, tests start on schedule (by a Windows scheduled tasks), and tests on these dispatchers run simultaneously. For Sikuli to work on each of those dispatchers, you need a Windows user to be logged in. So from my desktop, I have RDP sessions opened to each of those dispatchers. I open these sessions either via mstsc.exe (a standard Windows RDP connection), or with Microsoft Remote Desktop Connection Manager where you can open multiple RDP sessions within one snap-in. Here is the brief description of this tool: https://www.microsoft.com/en-us/download/details.aspx?id=44989

A question:

In the environment described above, when tests run on dispatchers, sometimes I observe mouse pointer moving on my desktop while Sikuli is being involved on one of the dispatchers. I can't say that it bothers me, but what I am really worried about is whether tests being used simultaneously on some dispatchers can affect each other. May it be possible that mouse clicks or keyboard typing would fail on one of the dispatchers if some mouse clicks or keyboard typing happens on another dispatcher at the same time? Is it possible, or mouse and keyboard using on separate dispatchers I have RDP sessions to are completely independent of each other?

Thank you!
Cedar

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
Cedar
Solved:
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said : 		#1

A SikuliX test case on one physical machine cannot be influenced by any other SikuliX stuff on any other machine - SikuliX itself does not have any feature to do that.

If you see any mouse movement on your master machine, then this is caused by the RDP stuff, depending on what you have setup on mouse and keyboard handling.

While a test runs in one of your dispatchers, you should see the screen changes in your respective RDP window. Wether you can see the mouse move, again depends on setup and speed aspects.

What you never should do, while a test runs on a dispatcher: do not do anything intentionally with mouse and keyboard on the master, while they are catched by the RDP window: this actions would go to the dispatcher and might interfere the test.

Revision history for this message
Cedar (cforest) said : 		#2

Thank you RaiMan!

Do I understand right that a SikuliX test case on one machine cannot be influenced by any other SikuliX stuff on another machine even if these machines are virtual ones running on remote VMware ESXi server, not physical ones?

As to mouse and keyboard on the master (physical desktop), I got it, no intentional actions with mouse and keyboard. At the same time, what about common activity? May I just type something in the editor, use mouse in the browser etc? Any chances that such actions would somehow affect SikuliX behavior on dispatchers?

Thanks,
Cforest

Revision history for this message
RaiMan (raimund-hocke) said : 		#3

at: Do I understand right that a SikuliX test case...
virtual or physical does not matter: the SikuliX features intended to act on mouse and keyboard of the machine, the script/test is running on, do not have any side effects towards other machines. So if the virtual machines itself do not do anything in this direction nor the script itself does it, the tests running on the dispatchers are completely independent from each other.

at: As to mouse and keyboard on the master (physical desktop)...
Yes, only mouse/keyboard actions inside a dispatcher RDP window might interfere a running test on the dispatcher. For all activities outside an RDP window on the master it is valid, that the master is only another machine like the dispatchers are: independent.

Happy testing

Revision history for this message
Cedar (cforest) said : 		#4

Thank you RaiMan!

Revision history for this message
Cedar (cforest) said : 		#5

Hi RaiMan,

Just a note in case it would be useful for anybody:

In the environment described above, I saw the following: when the script used Sikuli's paste function on one dispatcher, I did manual copy-paste in the command line on another dispatcher and guess what? On this second dispatcher, this manual paste operation returned the text from the first dispatcher (the text that the script used in the Sikuli's paste), not the text I copied on the second dispatcher.

I do realize that this issue is not directly related to Sikuli itself, but it is an issue people like me might bump into. Have no idea how to cope with it at the moment.

Thanks,
Cforest

Revision history for this message
Cedar (cforest) said : 		#6

So the only workaround I see at the moment is to disable copy-paste (clipboard) in RDP sessions (Google provides a couple of ways to do it - say unchecking Clipboard in RDP session' settings. As to whether or not mouse movements in one RDP session might affect another RDP session, I can't say at the moment - will need to observe for some time.

Revision history for this message
RaiMan (raimund-hocke) said : 		#7

--- As to whether or not mouse movements in one RDP session might affect another RDP session
... still no way, SikuliX can do that - be sure ;-)

--- So the only workaround I see at the moment is to disable copy-paste
paste() in SikuliX uses the SystemClipboard and a simulated ctrl/cmd-v afterwards. So if you have the Clipboard-transfer option on, then of course this can have side effects depending on your settings in your RDP environment.
So if you need paste() in your tests, then the action "disable copy-paste (clipboard) in RDP sessions" seems to be appropriate.

Revision history for this message
Cedar (cforest) said : 		#8

Thank you, Raiman!

Yes, I am absolutely sure about SikuliX in that sense!

I just worried about RDP stuff, because I observed mouse cursor movements on my desktop when Sikuli moved mouse cursor on the dispatcher. So I just thought something like "wow, processes in RDP session somehow affect the mouse cursor location on my desktop, and what if the mouse cursor on my desktop will in turn affect mouse cursor in another RDP session?"

I realize that such conclusions might look dummy, and I really hope they are dummy indeed, so I will take a look to be double sure :)

Take care,
Cforest

Revision history for this message
RaiMan (raimund-hocke) said : 		#9

happy looking ;-)