How to run "Sikuli On Virtual Machines"

Some people succeeded in using some VNC solution.

But as I understand, Sikuli does not work in any "headless" situation.

thankx RaiMan, i have also solved the problem.. but it took me almost a week to fix it!!

what i have observed that when you take rdp of 1 system(Lets suppose machine A) from another machine(Lets call it machine B, machine A gets locked!! so even when u release the rdp.. machine A is still locked!! so our sikuli wont run there!!

NOW install teamviewer 7 in both the machines A and B. and release the rdp... try to log into machine A from Machine B using teamviewer.. try it continously...

After some time , u ll see the screen of machine A from machine B in teamviewer.. unlock it.. and run the scripts.. we can even minimise the teamviewer windows but still sikuli works fine.. and in my case this machine A is a virtual and completly "headless" machine having windows OS...

Now what i have to do is extend this session of teamviewer between machine A and machine B, by generating some hardware level interrupts and also I have to prevent them from sleeping or going into a lock state!!

I ll get back to you with some more inputs.

Best Regards,

Anmol Bagga
Software Engineer - Test Automation

As I suggested in other related posts here, go with VNC if you must run Sikuli over remote machines.

RDP locks the desktop when you disconnect from RDP session, VNC doesn't.

As to Raiman's comment about "headless" mode issue, virtual machines are never headless. There's always a virtualized desktop head via the virtual machine manager (VMM). But it is often more efficient and convenient to access via RDP, VNC, SSH instead of using the virtualized desktop from the VMM interface. If you install a VNC server on the virtual machine, you will always get direct remote access to the "head" of the machine, as long as the networking stack of the machine and the VMM is working properly (or no network issues in the network).

And regarding locked desktops, you always want to be sure that the OS is configured to never lock itself when inactive or going to screensaver mode (and better yet, never go into screensaver mode, a pointless mode for virtual machines that don't have a "physical" head). But IT security policies might not allow it. And it can be tricky figuring out how to disable those auto-lock features when they're on by default (particularly when not configured by IT but came that way with the OS).

Yes mate, But we are getting complications setting/configuring up VNC server and VNC viewer due to network policies, firewalls etc..

we have prevented the Virtual machine from going into the locked state or running any kind of screensaver by generating a hardware level interrupt in every 59 seconds.

Thus we have used teamviewer but againg the maximum session of the team viewer(free version) is 30 minutes to 1 hour.. is there any way we can see(desktop share) the screen of virtual machine 24*7 on our machine.????

what exact issues are you having with VNC server & viewer/client setup? Perhaps you can work something out with IT to use a specific port for VNC. It defaults at 5900 for the server.

Or maybe tunnel through an accepted port like 80, etc. though I've never tried VNC tunneling, say over SSH or something else.

Also, if issue is with security, here are a few things to consider:

* can set VNC password to prevent unauthorized access. Free versions are not encrypted within the VNC protocol, I think. If need higher security, then buy the commercial versions. But even without encryption, the password will block off simple attempts at VNC hacking corporate machines. If the hacker managed to try sniffing your network for unecrypted passwords, then your network is already compromised, even if they didn't find any passwords.

* VNC is still decently secure as long as you run the machines within corporate network, not accessible externally. Therefore, to access them outside like working from home, you still need to VPN in.

* you could also secure down the test machines so even if hacked, limit the damage, like machines have restricted or no network fileshare access (besides HTTP web browsing).

And if VNC setup issue is with Windows Vista or newer OS, you may need to use specific software like UltraVNC or TightVNC. RealVNC freeware version doesn't seem to work with Windows 7, etc.

And why are you using TeamViewer? Is Microsoft RDP not working well enough for you? RDP will work fine as long as you keep the RDP window open and not close it. Though I still prefer VNC because you can close it and automation still runs.

I would say VNC should be your solution, you just need to get IT to agree to it somehow. Or live with the limitations of TeamViewer and RDP (keep session open, time limit, etc.)

ha ha.. u got me..

We are using window 7 on our machines , and the VM's os is windows server 2008.. most of the ports except some(8080 on which we are running fitnesse server) are blocked..

why we arent using RDP, because the VM gets locked and we cant run our test from anywhere, except the computer thru which we have taken RDP.

why we are using teamviewer because , then we can run our test from anywhere as the VM is running fitness server(on port 8080 , web service)... we can just use the ip+port and run our test from any machine!!

now what i have done is made a c++ program that will move the mouse pointer a pixel to its left after every 15 second..
and found a software that will generate keyboard press after every 59 second.. its name is caffeine!!
and extended the session to almost indefinite period..

Thanks
My problem is solved...

ha ha.. u got me..

We are using window 7 on our machines , and the VM's os is windows server 2008.. most of the ports except some(8080 on which we are running fitnesse server) are blocked..

why we arent using RDP, because the VM gets locked and we cant run our test from anywhere, except the computer thru which we have taken RDP.

why we are using teamviewer because , then we can run our test from anywhere as the VM is running fitness server(on port 8080 , web service)... we can just use the ip+port and run our test from any machine!!

now what i have done is made a c++ program that will move the mouse pointer a pixel to its left after every 15 second..
and found a software that will generate keyboard press after every 59 second.. its name is caffeine!!
and extended the session to almost indefinite period..

Thanks
My problem is solved...

anmol_bagga10, can you elaborate on a few things:

* do you keep TeamViewer open all the time or you connect and disconnect at times same way as with VNC?

* what port then does TeamViewer run on? Or I assume it also uses RDP and same ports.

I would also like to point out that you can run from anywhere using standard RDP provided the following:

* always connect to console session 0, either with RDP or mstsc.exe command line parameter "/admin" or "/console" exact parameter depends on version of RDP client and the OS connecting to. You can tell which session by checking under Task Manager users tab, displaying column for session ID.

* on client OSes, you will always get console session 0 (Win XP, and I think Vista and Win 7). On server OSes (2003, 2008), they give you free 3 user terminal server license. Hence when RDP'ing to a server OS, you will by default use session 1 or session 2. But there is always a session 0 (the head or desktop of server), that you can access via the parameter mentioned. Session 1 and 2 are virtual "terminal server" sessions.

With that in mind, if you connect to server A from machine B using session 0, you can disconnect and reconnect to same session 0 from machine C. That won't work if you connected via session 1 or 2, which are tied to client machine that did RDP. Session 0 is not as it is always the head/desktop. But unlike VNC, on disconnect of RDP, you get locked, because unlike VNC, physical desktop is not shared with RDP session is either/or, but VNC can use physical desktop and VNC session at same time (though not desirable as you be like fighting for mouse and keyboard). Monitor-wise VNC works like a splitter mirroring your desktop to a remote client unlike RDP.

@ anmol
--- now what i have done is made a c++ program that will move the mouse pointer .......
This could have been done in in the Sikuli script itself using sub processing or with standalone script, running in parallel.

And this is my script to prevent screen saver or sleep mode.

off = 1
stop = False
setShowActions(True)
while True:
    current = Env.getMouseLocation()
    x = current.x
    for i in range(3):
        if stop: break
        hover(current)
        wait(6)
        if abs(Env.getMouseLocation().x-x) > 0: stop = True
    if stop: break
    off *= -1
    go = current.offset(off, 0)
    hover(go)

Moving the mouse shortly after the slow motion animation will stop the script.

Sorry, forgot to mention: only for demo purposes.

For real life, this would be enough:

off = 1
while True:
    current = Env.getMouseLocation()
    x = current.x
    wait(30) # or whatever needed
    off *= -1
    go = current.offset(off, 0)
    hover(go)

But to end it, you have to kill the process.

now that I think about this...

wait() is in seconds right? As demo is fine, but for deployment should have a longer wait/sleep like min of 5 or 10 minutes up to an hour. If the background Sikuli script runs often then it may become CPU intensive.

Also sending keystrokes is probably a better option as mouse movements might hijack whatever automation is being run, if there are parallel tasks. Less conflict with keystrokes, although it may still interfere.

btw, my previous post was regarding any such background keep-alive automation, wasn't specific to Sikuli doing that.

I would think a good background process that does such keep alive should check if system is idle and only perform action when idle, not while some automated test is running. how to do that I don't know...

--- sending keystrokes is probably a better option
That is ok If you use a keystroke (combination), that does not do anything to the foreground application and does the keep-alaive job. I think, moving the mouse just one pixel usually does not harm much.

--- should check if system is idle
I do not think, that this is needed in this case of a keep-alive-task, since the resource usage is minimal.

Genrally, this approach can be optimized:
- self adjustment based on evaluation of the current timers for screensaver/sleep mode
- tracking mouse movement, to find out wether something needs to be done at all
- ...

This is an old thread, but I think this is the correct place to post this.

The following links are from another tool but they are about running tests using Remote Desktop and apply to all GUI automation:

http://support.smartbear.com/viewarticle/27110/

http://support.smartbear.com/viewarticle/30217/

http://support.smartbear.com/viewarticle/27448/

Hi all,
I find it impossible to run SikuliX tests (on a website opened in IE) through Jenkins on a RDC: I have the test scripted on the machine (B - win server) and when running them with the RDC open (connected from A - win 7) all works good, but if I close RDC and start the tests from Jenkins (bash command: runScript.cmd -r ) SikuliX is unable to find anything on the screen. I also tried installing TightVNC and oppening a session to B with it, but the error in jenkins is the same (FindFaied - as if the screen in black).
Can somebody explain how to fix this, so that I will be able to run tests from anywhere through Jenkins (and without having access (or opening a RDC) to the machine (B) that holds the tests)?

@Bogdan
Not yet really contained in my know-how-basket ;-)

but googling with
"sikuli" "jenkins"

reveals 90.000+ entries ;-)

Hi and thanks for replying,

I have searched the www and I have found many posts with Sikuli and Jenkins, but all are pointing to questions without answers (and this problem kind of passes over this integration problem)...

So far I have learned that if I connect trough RDC (in Remote Desktop Services Manager) the session is RDP-Tcp#0 and if I connect through VNC it is Console.
Now when the session is linked to the Console, my Sikuli (1.0.1, java version 1.6.0_33) test also fails. I have tried this from Jenkins (using tscon [sessionID] /dest:console) and from TightVNC (connecting directly, seeing the session is console, running SikuliIDE and starting the tests), both times Sikuli fails in finding... Is there any way making Sikuli work in a "console" Session?

Interesting, but as already mentioned: I have no experience with this.

I will give you a note, when the version 1.1.0-Beta1 is available: It contains a simple remote solution, that allows to run a script on one machine, while using screen, keyboard and mouse on another machine reachable via TCP/IP.

Hi,

I just ran across SikuliX and I am having fun playing around with creating scripts in the IDE for automating tasks. I am wondering about the possibility of using this over VNC or other remote connections and although I see a lot of relevant discussion I can't find any concrete examples.

I found this snippet of script on github but it doesn't seem to work in the IDE:
https://github.com/sikuli/sikuli/blob/develop/README.md

I get an error that the screen object has no connectVNC attribute. Is something like this still supported?

I also understand that the new version has a remote server implementation in Java (previous post). Is there also an API for accessing this functionality in the scripts that I can create in the IDE? I couldn't find anything in the documentation.

Or am I asking the wrong questions?

Thanks!
Andy

@Andy
--- the VNC solution with RC3 is not supported any more.
So you might have to live with the sources.

--- in version 1.1.0 there is a new VNC solution ...
... but this is not yet stable and under investigation ()

--- the remote server feature in 1.1.0 (setup option 5) ...
... can be used from the script level as well.
I will add some info to the docs.
meanwhile you have to look into the sources here as well.

> --- in version 1.1.0 there is a new VNC solution ...
> ... but this is not yet stable and under investigation ()

Do you have update on this ?
I have not found any doc :-(

>--- the remote server feature in 1.1.0 (setup option 5) ...
>... can be used from the script level as well.

Same question, I'm interested on this !

@RaiMan,

sorry, I didn't understand your answer for help about
"--- the remote server feature in 1.1.0 (setup option 5) ..."

Actual 1.1.0 setup have 3 options.
Does the "remote server" is deprecated ?

I came across this post a while back, forgot to mention here. Might be useful for those using Sikuli remotely with Remote Desktop:

http://www.allianceglobalservices.com/blog/executing-automation-suite-on-disconnectedlocked-machines

@daluu
Thanks for the pointer. I will have a look at it and probably support it in SikuliX version 2.

Hi,
I have a remote machine R where i have scheduled a few automated runs using sikuli .
Issue: My script starts at the decided time on R but as soon the code flow come at reading UI element/search image pattern , the sikuli script does nothing. it stales completely so the only way i am able to run it is by opening up "mstsc R" on my workstation and then keep that mstsc window open .

Can someone please advise on this ?

Thanks in advance.

@RaiMan , can you please advise ?

I do not have any experience with that, sorry