SikuliX

FindFailed with VNCScreen

Asked by Aditya Basak

So, I have an image stored at a location "D:\\example.png" on my local computer.
I wish to perform a click on the part of the screen on my remote (here virtualBox) computer screen, which matches with "example.png".
This remote PC has a VNC Server (TigerVNC) installed and running on port 9876.
My connection to the VNC Server is working through Eclipse code, but the find is failing, although physically the match of "example.png" is available on the page.

I tried decreasing the similarity quotient but yet it fails.

Here is my code. :

VNCScreen client = VNCScreen.start(<ip-address>, <port>, <password>, 10, 1000);
try {
 client.click(new Pattern("D:\\example.png").similar(0.5f));
 }
catch (FindFailed e) {
 e.printStackTrace();
 }
finally {
 client.close();
 }

The error message is :

CConnection: Using RFB protocol version 3.8
R[0,0 1024x768]@S(0) E:Y, T:3.0
FindFailed: D:/example.png: (84x31) in R[0,0 1024x768]@S(0) E:Y, T:3.0
  Line 2782, in file Region.java

So next to debug, I tried capturing the screen that Sikuli sees on connecting with the VNC with the following code :

ScreenImage image = client.capture(client);
 image.save("D:\\");

The saved image, is a complete black screen with no items.
Can you point where I am going wrong?

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
RaiMan
Solved:
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said : 		#1

Using latest SikuliX 1.1.3?

Running on what system?

Revision history for this message
Aditya Basak (aditya94) said : 		#2

Yes. Using Sikuli 1.1.3

Both the systems are on Windows 10. The code is written in Eclipse Oxygen.

Revision history for this message
RaiMan (raimund-hocke) said : 		#3

Do you get content with a VNCViewer?

Revision history for this message
Aditya Basak (aditya94) said : 		#4

Yes. VNCViewer works fine. Not only do I get content, but I can work pretty well on the remote.

Revision history for this message
RaiMan (raimund-hocke) said : 		#5

ok, thanks for information.

similar problem here:
https://answers.launchpad.net/sikuli/+question/669590

... as mentioned there: takes some time for me to check.

Until I (may be) fixed it: what about working against the VNCViewer window as workaround?

Revision history for this message
Best RaiMan (raimund-hocke) said : 		#6

made a quick test in max local environment:
Server: TightVNC on a Windows 10 machine
Client: Mac running a script with latest 1.1.3

# script:
vs = vncStart("192.168.2.120")

if not vs:
  print "no connection"
  exit()

wait(1)
cap = vs.userCapture("gimmi")
img = "_vncscreen"
cap.saveInBundle(img)

vs.hover(img)
off()

My findings:
Even in my local network, the initial fill of the frame buffer after the VNCScreen.start() takes about 1 second.
So without the wait(1) I get a black screen with the userCapture(), with it, the screen content is correct.

Doing a search with an image directly after the VNCScreen.start() works for me, because of the standard wait time of 3 seconds (in my case doing the vs.hover(img) directly after start)

so you might try this:
VNCScreen client = VNCScreen.start(<ip-address>, <port>, <password>, 10, 1000);
try {
 //client.click(new Pattern("D:\\example.png").similar(0.5f));
    client.click(client.wait("D:\\example.png", 10));
 }
catch (FindFailed e) {
 e.printStackTrace();
 }
finally {
 client.close();
 }

... so adding some time to wait for the frame buffer to get filled the first time.

Revision history for this message
Aditya Basak (aditya94) said : 		#8

Thanks RaiMan, that solved my question.