after sikulix script completes my mouse clicks do not work right via TightVNC

Asked by Doug Symes

I'm launching my jar from java within a windows batch file that is called by windows task scheduler. However, after the scheduled task completes and the Windows CMD Prompt closes, then if I try to click on a specific task in Windows Tasks list, all the tasks I click on highlight. Its as if I've anchored the first task and am using SHIFT+Click to select several tasks. If I close TightVNC and reopen it all is well. I checked the Keyboard mapping and it is the normal US English (00000409). Please advise.

Question information

Language:
English Edit question
Status:
Answered
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said :
#1

should first be a question

Revision history for this message
RaiMan (raimund-hocke) said :
#2

So it seems you are using VNCScreen?

Or are you running SikuliX scripts on a remote machine, that is managed via a TightVNC session?

Revision history for this message
Doug Symes (dsymes) said :
#3

I am running SikuliX scripts on a remote machine, that is managed via a TightVNC session.

Windows Task Scheduler calls the batch file that has this code:

start /i /b /wait c:\users\biuser\desktop\sikuli\runsikulix -f c:\users\biuser\desktop\sikuli\sikuli-log.txt -r c:\users\biuser\Documents\QMG_Live.sikuli
findstr /i /g:c:\users\biuser\desktop\sikuli\log_string_search_list.txt c:\users\biuser\Desktop\Sikuli\sikuli-log.txt
if %errorlevel% == 0 (
   GOTO SENDMAIL
) ELSE (
   GOTO NOTHING
)
:SENDMAIL
 aws ses send-email --to <email address hidden> --from <email address hidden> --source-arn arn:aws:ses:us-east-1:<acct>:identity/qualbe.com --subject "Data-Mining BI Error" --text "Please login to data-mining.qmg.local and check on PBI Desktop to make sure its still running okay. Look at the log file from the automated Sikuli Windows Scheduled Task. The log is located here: c:\users\biuser\Desktop\Sikuli\sikuli-log.txt"
:NOTHING
exit
------ EOF -----

1) What do you make of the VNC issue?
2) Is the above batch file a typical use for SikuliX / Acceptable use?

Revision history for this message
RaiMan (raimund-hocke) said :
#4

Ok, I understand:
- there is a SikuliX environment on the remote machine
- there is a script .sikuli on the remote machine
- also on the remote machine the scriptrun is started via Task Scheduler
- ... and might send an email after termination based on an evaluation of the SikuliX log info

--- What do you make of the VNC issue?
the SikuliX run does not know anything about the fact, that you look at the remote screen using a VNC session.
Hence your problem does not have anything todo with SikuliX, but maybe with your script.

Does the script use the type() feature of SikuliX or even keyUp/keyDown?

--- Is the above batch file a typical use for SikuliX / Acceptable use?
Typical: I do not know.
Acceptable: yes. From SikuliX's point of view it simply runs on a capable machine.

Revision history for this message
Doug Symes (dsymes) said :
#5

I only use type() for hitting ESC b/c at one point when I'm clicking on things in the application, the screen switches as if someone hit the Windows key (this could happen in any of several places) so I inserted a check for it and hit ESC to return to the application.
Here is the whole PY file from SikuliX - as you can see I'm almost always looking for images exclusively. I will tack on the log I'm getting periodically for your perusal:

#Launch PowerBI and open the QMG-Live PBIX file.
doubleClick(Pattern("1544653159352.png").similar(0.82))
wait(Pattern("1544656197122.png").similar(0.61),60)
click(Pattern("1544215649288.png").similar(0.80))
if exists(Pattern("Server_Desktop.png").similar(0.80)):
    type(Key.ESC)
if exists(Pattern("1544215829534.png").similar(0.74),5):
    click(Pattern("1544215896229.png").similar(0.89),5)
if exists(Pattern("Server_Desktop.png").similar(0.80)):
    type(Key.ESC)
waitVanish(Pattern("1544216116787.png").similar(0.80),80)
if exists(Pattern("Server_Desktop.png").similar(0.80)):
    type(Key.ESC)
wait(Pattern("1544216171532.png").similar(0.57),20)
if exists(Pattern("Server_Desktop.png").similar(0.80)):
    type(Key.ESC)
click(Pattern("1544216198461.png").similar(0.80),20)
if exists(Pattern("Server_Desktop.png").similar(0.80)):
    type(Key.ESC)
if exists(Pattern("1544824408767.png").similar(0.80),10):
    click(Pattern("1544826489006.png").similar(0.80),5)
else:
    if exists(Pattern("Server_Desktop.png").similar(0.80)):
       type(Key.ESC)
    if exists(Pattern("1544718781015.png").similar(0.80),1):
       click(Pattern("1544718781015.png").similar(0.80),1)
if exists(Pattern("Server_Desktop.png").similar(0.80)):
    type(Key.ESC)
Found = False
i=0
while i < 5 :
    i = i+1
    if i>1:
       if exists(Pattern("1544718936612.png").similar(0.80),5):
          click(Pattern("1544718936612.png").similar(0.78),5)
    if exists(Pattern("1544719057841.png").similar(0.82),5):
        Found = True
        break
if exists(Pattern("Server_Desktop.png").similar(0.80)):
    type(Key.ESC)
if Found:
    click(Pattern("1544719165835.png").similar(0.78),6)
    if exists(Pattern("Server_Desktop.png").similar(0.80)):
        type(Key.ESC)
    click(Pattern("1544719189664.png").similar(0.86),6)
    if exists(Pattern("Server_Desktop.png").similar(0.80)):
        type(Key.ESC)
    if exists(Pattern("1544824573671.png").similar(0.80),6):
        click(Pattern("1544719214036.png").similar(0.82),6)
    if exists(Pattern("Server_Desktop.png").similar(0.80)):
        type(Key.ESC)
    wait(Pattern("1544719259439.png").similar(0.78),180)
    if exists(Pattern("Server_Desktop.png").similar(0.80)):
        type(Key.ESC)
    click(Pattern("1544719321513.png").similar(0.75),5)
    if exists(Pattern("Server_Desktop.png").similar(0.80)):
        type(Key.ESC)
    if exists(Pattern("1544719633417.png").similar(0.80),5):
        click(Pattern("1544719633417.png").similar(0.80),5)
    if exists(Pattern("Server_Desktop.png").similar(0.80)):
        type(Key.ESC)
    if exists(Pattern("1544826115697.png").similar(0.79),3):
        click(Pattern("1544826115697.png").similar(0.79),3)
if exists(Pattern("1544719633417.png").similar(0.80),3):
    click(Pattern("1544719633417.png").similar(0.80),3)
if exists(Pattern("1544826115697.png").similar(0.79),3):
    click(Pattern("1544826115697.png").similar(0.79),3)
#the end

=================== Sikuli Log File (usually there are not errors) ===============================
[error] script [ C:\Users\biuser\Documents\QMG_Live.sikuli ] stopped with error at line --unknown--
[error] Error caused by: Traceback (most recent call last):
  File "C:\Users\biuser\Documents\QMG_Live.sikuli\QMG_Live.py", line 2, in <module>
    doubleClick(Pattern("1544653159352.png").similar(0.82))
  Line 2761, in file Region.java

 at org.sikuli.script.Region.wait(Region.java:2761)
 at org.sikuli.script.Region.find(Region.java:2326)
 at org.sikuli.script.Region.getLocationFromTarget(Region.java:3220)
 at org.sikuli.script.Region.doubleClick(Region.java:3767)
 at org.sikuli.script.Region.doubleClick(Region.java:3752)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
 at java.lang.reflect.Method.invoke(Unknown Source)
org.sikuli.script.FindFailed: FindFailed: 1544653159352.png: (68x89) in S(0)[0,0 1920x1080] E:Y, T:3.0
  Line 2761, in file Region.java

Revision history for this message
RaiMan (raimund-hocke) said :
#6

I am sorry, but I have no idea, what is happening in your case.

BTW: are you using the latest build of 1.1.4?

Revision history for this message
Doug Symes (dsymes) said :
#7

1.1.3 SikuliX
Java 1.8.0.191

What is the upgrade process?

Revision history for this message
Doug Symes (dsymes) said :
#8

In addition to finding out the upgrade process to go from SikuliX 1.1.3 to 1.1.4;
Can you tell me how you interpret this log data?

[error] script [ C:\Users\biuser\Documents\QMG_Live.sikuli ] stopped with error at line --unknown--
[error] Error caused by: Traceback (most recent call last):
  File "C:\Users\biuser\Documents\QMG_Live.sikuli\QMG_Live.py", line 2, in <module>
    doubleClick(Pattern("1544653159352.png").similar(0.82))
  Line 2761, in file Region.java

 at org.sikuli.script.Region.wait(Region.java:2761)
 at org.sikuli.script.Region.find(Region.java:2326)
 at org.sikuli.script.Region.getLocationFromTarget(Region.java:3220)
 at org.sikuli.script.Region.doubleClick(Region.java:3767)
 at org.sikuli.script.Region.doubleClick(Region.java:3752)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
 at java.lang.reflect.Method.invoke(Unknown Source)
org.sikuli.script.FindFailed: FindFailed: 1544653159352.png: (68x89) in S(0)[0,0 1920x1080] E:Y, T:3.0
  Line 2761, in file Region.java
========================== EOF =================================

Is it as simple as Whatever is on line 2 had an error? Is it that the image listed on the last line of the log was just not found on the screen and the code timed out?

Revision history for this message
RaiMan (raimund-hocke) said :
#9

There is no update process - simply follow the steps on the download page.

--- FindFailed: exactly

You might read the docs, to finally understand and see other possibilities.

Revision history for this message
Doug Symes (dsymes) said :
#10

I'll lean in and see if I can't figure this out. Thanks much.

Can you help with this problem?

Provide an answer of your own, or ask Doug Symes for more information if necessary.

To post a message you must log in.