exists() but cannot click()

Surely not a bug.

Generally one should use a match, that was found before, instead of searching again.

while exists(image, 60):
  click() # will click the match found by exists
  wait(image1, 30)
  click()

Your problem might be caused by timing: it takes rather long to search the whole. So the target might have vanished again or changed a bit on the screen (too much background in the shot ??)

The only real way to prove that the image changed would be to capture the screen at end of while loop and before the click. I have been unable to locate methods to accomplish the captures which was one reason I posted this incident. I have seen repeats of this behavior in other sections of code. The application that I am manipulating does not update its image except in reaction to events such as keystrokes or mouse clicks so while one can claim that the image must have changed, I am reluctant to believe it given how the application behaves. And without a method to capture the screen, I cannot provide evidence to convince myself that the screen changed. In addition, I have seen similar code fail with a different application that updates the screen even less frequently than the one referenced in this report.

    mainWin.click("1392777214447.png") # click on accept to proceed with install in Google play store
    mainWin.wait("1395278354165.png",60) # wait for proceed message to use wifi to download large app
    print 'click proceed to use wifi'
    mainWin.click("1395278354165.png") # click on proceed

this is a code segment that failed in similar manner, the wait was for the warning message that were download a very large app and to proceed using wifi only. In the failure I saw, the second click failed after the wait was successful. The screed at time of the failure clearly had the proceed button.

I was unaware that click() cated on the last exists() or find() that occurred as that behavior was not documented in the Sikuli documentation. That simple fact will speed up my code significantly. Thanks for that tip.

I agree that searching screen can take a while so I have been converting code to limit search areas. My current methodology to increase speed has been to define regions at location where particular images will appear and check for those images in regions the same size as the image.

aButton = find("abutton.png")

:
:
:
if aButton.exists("abutton.png"):
    #button is there click it
    aButton.click()

to capture screen content at any time:
img = capture(SCREEN) # whole screen
img = capture(some_region) # the given region only

img now is a temp filename.

to keep it:
import shutil
shutil.move(img, "some absolute path.png")

I added some screen captures and have proven to myself that sikuli gets 'lost' after processing for a bit. While I did not capture the initial failure image, I did capture an image after restarting a script. I watched as the mouse arrow remained motionless as the script ran. After a short while the script fails.

code snippet
    if mainWin.exists("1395554772408.png"): #captured imaged shows this button in both fullscreen and mainWin captures
        print 'click uninstall'
        mainWin.click("1395554772408.png") # mouse did not move to this so could never have clicked it.
        #wait(3)
        print 'click ok to uninstall'
        mainWin.wait(Pattern("1392777180952.png").similar(0.90),30) # without above click, this wait should fail it didn't
        mainWin.click(Pattern("1392777180952.png").similar(0.90))
    print 'mainWin =',mainWin
    beforeInstall = capture(mainWin)
    print 'mainWin captured before install to',beforeInstall
    beforeFSInstall = capture(fullscreen)
    print 'fullscreen captured before install to',beforeFSInstall
    mainWin.wait(Pattern("1395554852794.png").similar(0.90),60) # this failed because image not on screen

the log ...

click uninstall

[log] CLICK on L(1112,203)@S(0)[0,0 5760x1080]

click ok to uninstall

[log] CLICK on L(1462,870)@S(0)[0,0 5760x1080]

mainWin = R[0,0 1920x1080]@S(0)[0,0 5760x1080] E:Y, T:3.0

mainWin captured before install to C:\Users\kris\AppData\Local\Temp\sikuli-2040819777259839293.png

fullscreen captured before install to C:\Users\kris\AppData\Local\Temp\sikuli-1448569504283501216.png

[error] script [ pass4 ] stopped with error in line 1904
[error] FindFailed ( can not find P(1395554852794.png) S: 0.9 on the screen. )

the two clicks NEVER happened as I wattched the mouse sit motionless during the execution. the images captured show the image file from the first click on the screen.

there is obviously an aging issue here. Once it is in this state, further executions result in repeats of the same failure. However, if shutdown and restart the script, it works correctly.

Obviously this cannot be debugged from my log file. I need a code snippet that will turn on and turn off whatever failure data capture features are in sikuli/jython/java. Adding code and rerunning causes code to work correctly again so the capture needs to be there until the failure. (I have to assume that pressing run does not regenerate bytecode unless scripts have been changed.)

I understand:
The problem only happens, when you rerun the script in the IDE.
It does not happen, when start the IDE from scratch and run your script the first time

If this is true and it only happens on restart, then the problem cause must be after the above snippet is processed the first time.

BTW: What happens in this case?
   if mainWin.exists("1395554772408.png"):
        print 'click uninstall'
        mainWin.click()

On the other hand: If the image is there and found (which obviously is the case, since no FindFailed Exception with the
mainWin.click("1395554772408.png") ) then it is some mouse blocking problem.

One more thing:
After the above code snippet, do you change the FindFailedResponse settings somehow?
Currently not everything is correctly reset on rerun in the IDE.

Second more thing:
when running the script from command line, then it should always work.

While logfile I appended earlier was from a launch of the sikuli script after a failure, I have seen this failure before when started from the IDE as first operation. That is why I added the screen capture code to that snippet. So, to respond to the first statement, it happened when script was first run. I should point out that the script had been running for 2-3 hours before it failed. I have seen the exists() but not able to click() in a number of places in the code, so it is not limited to that section.

Agreed, sometime after the first invocation of that code or similarly failing other code something goes wrong.

When I tried using the click() after an exists() that worked, I did not get the behavior that you expected. The object that was in the exists was not clicked, I put off looking at it to find a simpler case. Based on brief other experiment, I expect it would fail also.

There has to be more than a mouse blocking issue because the second wait() and click() should have failed with not found since that popup message is dependent on clicking the install button which never happened.

        mainWin.wait(Pattern("1392777180952.png").similar(0.90),30) # without above click, this wait should fail it didn't
        mainWin.click(Pattern("1392777180952.png").similar(0.90))

While the print statements are operating, it appears that the wait and click statements are not processing. The wait() should have failed with not found, and the click() should have also failed with not found. The log contains a click() that never should have happened.
[log] CLICK on L(1462,870)@S(0)[0,0 5760x1080]

The code does not change the default behavior for the FindFailedResponse, I expect exceptions to be thrown in every case.

I will run the script from command line and see if similar failure occurs. Will add note with additional success/failure information. I will also try writing a simpler testcase to try to recreate.

Having stumbled on your webpage, I would tell you to enjoy your vacation and not worry about this. It can wait till April or later, please do enjoy yourself.

vacation: this part of my "Sikuli-slow-down" is over now, but I really had a rest. thanks.
now the other aspects force me to keep my Sikuli activities on a low level.

... but for such odd situations like yours, there should be enough time.

Your screen has a width of 5760: Is it a "virtual screen" defined over 2 monitors?
If yes, how is it setup in Windows?

Are the failing things in the same place as when not failing?

Run the script from command line using
runIDE -d 3 -r absolute-path-to-script.sikuli

you will get some more debug output.

I was able to get a failure running from the command line, here is complete log of failure with some sensitive information deleted.

C:\sikuli>runide.cmd -r c:\sikuli\scripts\pass4
+++ running this Java
java version "1.7.0_51"
Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)
+++ trying to start Sikuli IDE
+++ using: -Xms64M -Xmx512M -Dfile.encoding=UTF-8 -Dsikuli.FromCommandLine -jar
C:\sikuli\sikuli-ide.jar -r c:\sikuli\scripts\pass4
Mar 25, 2014 3:41:25 PM java.util.prefs.WindowsPreferences <init>
WARNING: Could not open/create prefs root node Software\JavaSoft\Prefs at root 0
x80000002. Windows RegCreateKeyEx(...) returned error code 5.
utf8
USERID = >>xxxxxxxxxxxxxx<< PASSWORD = >>xxxxxxxxxxxx<<
:
only thing not displayed were print statements
:
all done next stuff
idctr = 22
statdata =>>>>> <<<<<<
statdata = null True
statdata = space False
empty file
id found was xxxxxxxxxxxxxxxxx position = 0
mainWin captured before uninstall to C:\Users\kris\AppData\Local\Temp\sikuli-165
8904011163285165.png
click uninstall
[log] CLICK on L(1108,203)@S(0)[0,0 5760x1080]
mainWin captured before oking uninstall to C:\Users\kris\AppData\Local\Temp\siku
li-7956444703644258655.png
click ok to uninstall
[log] CLICK on L(995,492)@S(0)[0,0 5760x1080]
mainWin = R[0,0 1920x1080]@S(0)[0,0 5760x1080] E:Y, T:3.0
mainWin captured before install to C:\Users\kris\AppData\Local\Temp\sikuli-82884
61282641491864.png
fullscreen captured before install to C:\Users\kris\AppData\Local\Temp\sikuli-46
50763533158311038.png
mainWin.getLastMatch() = M[1174,185 147x42]@S(S(0)[0,0 5760x1080]) S:1.00 Target
:1247,206
click install
[log] CLICK on L(1247,206)@S(0)[0,0 5760x1080]
mainWin.getLastMatch() = M[1174,185 147x42]@S(S(0)[0,0 5760x1080]) S:1.00 Target
:1247,206
mainWin.getLastMatch() = M[888,677 139x41]@S(S(0)[0,0 5760x1080]) S:1.00 Target:
957,697
click accept
[log] CLICK on L(957,697)@S(0)[0,0 5760x1080]
mainWin.getLastMatch() = M[888,677 139x41]@S(S(0)[0,0 5760x1080]) S:1.00 Target:
957,697
screen captured before exists to C:\Users\kris\AppData\Local\Temp\sikuli-65370
98895274661841.png
mainWin.getLastMatch() = M[888,677 139x41]@S(S(0)[0,0 5760x1080]) S:1.00 Target:
957,697
screen captured after exists to C:\Users\kris\AppData\Local\Temp\sikuli-656192
427479170894.png
click proceed to use wifi
[error] script stopped with error in line 1929
[error] FindFailed ( can not find P(1395278354165.png) S: 0.9 on the screen. )
[error] --- Traceback --- error source first
line: module ( function ) statement
632: main ( installFromPlay ) mainWin.click(Pattern("1395278354165.png").si
milar(0.90))
[error] --- Traceback --- end --------------

C:\sikuli>

the code that was executing

608 print 'mainWin =',mainWin
609 beforeInstall = capture(mainWin)
610 print 'mainWin captured before install to',beforeInstall
611 beforeFSInstall = capture(fullscreen)
612 print 'fullscreen captured before install to',beforeFSInstall
613 mainWin.wait(Pattern("1395554852794.png").similar(0.90),60) #install button in playstore
614 print 'mainWin.getLastMatch() =',mainWin.getLastMatch() #code added to check where click() would go
615 print 'click install'
616 mainWin.click(Pattern("1395554852794.png").similar(0.90)) #install button
617 print 'mainWin.getLastMatch() =',mainWin.getLastMatch()
618 #wait(3)
619 mainWin.wait(Pattern("1395278296489.png").similar(0.90),60) #waiting for accept button
620 print 'mainWin.getLastMatch() =',mainWin.getLastMatch()
621
622 print 'click accept'
623 mainWin.click(Pattern("1395278296489.png").similar(0.90))
624 print 'mainWin.getLastMatch() =',mainWin.getLastMatch()
625 beforeProceed = capture(fullscreen)
626 print 'screen captured before exists to ',beforeProceed
627 mainWin.exists(Pattern("1395278354165.png").similar(0.90),60)
628 print 'mainWin.getLastMatch() =',mainWin.getLastMatch()
629 afterProceed = capture(fullscreen)
630 print 'screen captured after exists to ',afterProceed
631 print 'click proceed to use wifi'
632 mainWin.click(Pattern("1395278354165.png").similar(0.90))

things are even stranger now. none of the files listed for captured images exist.
the mouse never moved, nothing was clicked which means
the click on 616 did not happen which means the accept button did not appear
the wait on 619 should have failed
the click on 623 could not happen
the exists on 627 should have failed
the lastmatch did not change from previous value!
finally the click on 632 did fail

the displays are hooked up to a AMD Radeon 7850. a mini display port has an Accell MST multi-display hub.
three ordinary HDMI cables run from hub to each monitor. The software that controls this is the
AMD Catalyst Control Center. The three monitors together make up an Eyefinity Display group.
The display group presents itself as a single 5760x1080 display to windows.
sikuli's view
print 'getNumberScreens() = ',getNumberScreens
print 'getBounds() = ',getBounds()

getNumberScreens() = <bound method type.getNumberScreens of <class 'sikuli.Screen.Screen'>>
getBounds() = java.awt.Rectangle[x=0,y=0,width=5760,height=1080]

Next run will be from command with -d 3 ... will add comment it/when it fails

BTW, card is amazing. I get 30+ fps in warthunder at 'movie' resolution. card is rated at 800 gigaflops!!

--- things are even stranger now. none of the files listed for captured images exist.
these temp image files are purged at script termination.

To preserve them you need extra code

def myCap(msg, reg = SCREEN):
     import shutil
     cap = capture(reg)
     print msg, cap
     shutil.move(img, "absolute-path-to-safe-place-" + msg + ".png") # msg should not contain blanks or other special character

now instead of e.g.
beforeInstall = capture(mainWin)
print 'mainWin captured before install to',beforeInstall

use
myCap("mainWin_before_install", mainWin)

--- the exists on 627 should have failed
exists() cannot "fail", it simply returns the match if successful or null otherwise. So to know about the result of an exists, you have to check the return value
if exists(): print "found"
else: print "not found"

--- the lastmatch did not change from previous value!
this is by definition: only a find success changes the value, which is helpful in some situations (multiple exists on same region)

If you produce a log with -d 3, I would prefer to get a file instead of the paste here (silently malt o my mail at https://launchpad.net/~raimund-hocke)

BTW: what happens, if you isolate the above snippet into a separate script, drive your app manually to that point and then run the script?

I have been attempting to recreate with the -d 3 without any success so far. Is just like work, adding tracing or debugging code seems to change the environment sufficiently that the failure am trying to recreate does not happen.

will keep trying to recreate with -d 3