[1.0.1] SikuliX-API (sikuli-java.jar): problem after upgrading from 0.10.2

Asked by blackheat

I had ever written some sikuli script which using the sikuli-script-0.10.2.jar and it works well. Now I find sikuli-script was removed from maven repo so I turned to use the sikuli-java.jar introduced by raiman.

However, after the upgrade, I find the "find"method does not work any more. I write a simple program to find a picture on desktop and sikuli can't find that at all. The code is like below:

    public void findSomethingTest() throws Exception {

     Screen s = new Screen();
     s.setAutoWaitTimeout(10);
     System.out.println(s);
     Pattern p = getPattern(IMAGE_ROOT+"pps.PNG");
     Match m = s.find(p);
     m.highlight();

    }

the script will end with findfailure exception after 10 seconds. The error message is like below:

FindFailed: can not find P(-- BufferedImage --) S: 0.7 on the screen.
  Line 1574, in file Region.java

I am sure that p will get correct value and the pps.PNG is really on the desktop. It is just like that the image recognition does not work any more. Have any other experience with this?

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
RaiMan Edit question
Solved by:
RaiMan
Solved:
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said :
#1

What version do you use?

If you want to work with Sikuli, you have to start here for the current version 1.0.1 and use setup option 4 to get a valid sikuli-java.jar
http://www.sikuli.org/download.html

relevant information on using SikuliX 1.0.1 with Java:
https://github.com/RaiMan/SikuliX-API/wiki/Usage-in-Java-programming

If you already did that:

At the very beginning insert
Debug.setDebugLevel(3)

use an image, that surely is visible on the screen and change
s.setAutoWaitTimeout(2);

... so it crashes faster ;-)

and paste the complete output here.

only for completeness:
paste the getPattern function too

Revision history for this message
blackheat (iamblackheat) said :
#2

I choose the setup option as 3 during installlation of 1.0.1? Can I run setup again or I have to uninstall it and install again?

I am current outside, will try to use the Debug info and paste the getPatter(). However, there is little chance for bug in getPatter(). These are the scripts I developed before and runs well in sikuli-script-0.10.2

Revision history for this message
RaiMan (raimund-hocke) said :
#3

please empty the setup folder except sikuli-setup.jar and then run setup again with option 4.

send the setup log file silently to my mail at
https://launchpad.net/~raimund-hocke

another option of course is to try with RC3
http://www.sikuli.org/downloadrc3.html

Revision history for this message
RaiMan (raimund-hocke) said :
#4

I just tested and had no problems.

But by accident I found out, that when using a libs folder, that is not compatible to 1.0.1, I got the effect you are complaining about.

So be sure, that you use a libs folder, that is created either by setup or when running a Java program, that uses sikuli-java.jar.

You might check this by deleting the libs folder in the setup folder. It should be recreated and used at runtime.

Using the above mentioned Debug setting should tell you more about that.

Revision history for this message
blackheat (iamblackheat) said :
#5

Here comes the debug log:
okt 15, 2013 10:53:52 EM java.util.prefs.WindowsPreferences <init>
WARNING: Could not open/create prefs root node Software\JavaSoft\Prefs at root 0x80000002. Windows RegCreateKeyEx(...) returned error code 5.
[debug] Screen: initScreens: basic initialization (1 Screen(s) found)
[debug] *** monitor configuration (primary: 0) ***
[debug] Screen 0: S(0)[0,0 1366x768]
[debug] *** end monitor configuration ***
S(0)[0,0 1366x768] E:Y, T:3,0
class com.blackheat.sikuli.pps.Test1
file:/C:/Users/helen/workspace/testpps1/target/classes/images/pps.PNG
[debug] waiting for P(-- BufferedImage --) S: 0.7 to appear
[debug] Screen.capture: java.awt.Rectangle[x=0,y=0,width=1366,height=768]
[debug] RobotDesktop: captureScreen: on 0 using java.awt.Rectangle[x=0,y=0,width=1366,height=768]
[debug] ResourceLoaderBasic: SikuliX Package Build: 1.0.1 12SEP2013135849
[debug] ResourceLoaderBasic: check: we are running on arch: amd64
[debug] ResourceLoaderBasic: check: using Java at: C:/Program Files/Java/jre7/
[debug] ResourceLoaderBasic: check: Exists libs folder at location of jar? YES: D:/sikuliX/
[debug] ResourceLoaderBasic: checkLibsDir: D:/sikuliX/libs
[debug] ResourceLoaderBasic: loadLib: WinUtil
[debug] ResourceLoaderBasic: loadLib: Found: WinUtil
[debug] ResourceLoaderBasic: loadLib: Now loaded: WinUtil
[debug] ResourceLoaderBasic: checkLibsDir: Using libs at: D:\sikuliX\libs
[debug] ResourceLoaderBasic: check: Using this as OCR directory (tessdata) too
[debug] ResourceLoaderBasic: loadLib: VisionProxy
[debug] ResourceLoaderBasic: loadLib: Found: VisionProxy
[debug] ResourceLoaderBasic: loadLib: Now loaded: VisionProxy
[debug] Screen.capture: java.awt.Rectangle[x=0,y=0,width=1366,height=768]
[debug] RobotDesktop: captureScreen: on 0 using java.awt.Rectangle[x=0,y=0,width=1366,height=768]
[debug] Screen.capture: java.awt.Rectangle[x=0,y=0,width=1366,height=768]
[debug] RobotDesktop: captureScreen: on 0 using java.awt.Rectangle[x=0,y=0,width=1366,height=768]
[debug] Screen.capture: java.awt.Rectangle[x=0,y=0,width=1366,height=768]
[debug] RobotDesktop: captureScreen: on 0 using java.awt.Rectangle[x=0,y=0,width=1366,height=768]
[debug] Screen.capture: java.awt.Rectangle[x=0,y=0,width=1366,height=768]
[debug] RobotDesktop: captureScreen: on 0 using java.awt.Rectangle[x=0,y=0,width=1366,height=768]
[debug] Screen.capture: java.awt.Rectangle[x=0,y=0,width=1366,height=768]
[debug] RobotDesktop: captureScreen: on 0 using java.awt.Rectangle[x=0,y=0,width=1366,height=768]
[debug] Screen.capture: java.awt.Rectangle[x=0,y=0,width=1366,height=768]
[debug] RobotDesktop: captureScreen: on 0 using java.awt.Rectangle[x=0,y=0,width=1366,height=768]
[debug] Screen.capture: java.awt.Rectangle[x=0,y=0,width=1366,height=768]
[debug] RobotDesktop: captureScreen: on 0 using java.awt.Rectangle[x=0,y=0,width=1366,height=768]
[debug] P(-- BufferedImage --) S: 0.7 has not appeared.

and the dependency part in pom file for maven

  <dependency>
   <groupId>org.sikuli</groupId>
   <artifactId>sikuli-java</artifactId>
   <version>1.0.1</version>
   <scope>system</scope>
   <systemPath>\\D:\sikuliX\sikuli-java.jar</systemPath>
   <type>jar</type>
  </dependency>

and the definition of getPattern()

    private Pattern getPattern(String fileName) throws IOException {
        System.out.println(getClass());
        System.out.println(getClass().getResource(fileName));
        BufferedImage image = ImageIO.read(getClass().getResource(fileName));
        Pattern p = new Pattern(image);
        return p;
    }

Revision history for this message
RaiMan (raimund-hocke) said :
#6

see direct communication.

Seems that libs folder was not deleted before repeating the setup.

Revision history for this message
blackheat (iamblackheat) said :
#7

and will you look at the mail in night? I have to work full time so I can
only try the new suggestion in the night after my kid got sleep.so the
interaction between you and me seems too slow even we are in some time
zone( I am living in Stockholm )

by the way, I am developing using eclipse and maven. the OS is windows 8. I
can also use windows 7 but that one has installed sikuli before so I am not
sure the new installation will overwrite the old one or sikuli will got
strange behaviour if both version exist together.

Br/Dongbo

2013/10/16 RaiMan <email address hidden>

> Your question #237353 on Sikuli changed:
> https://answers.launchpad.net/sikuli/+question/237353
>
> Status: Open => Answered
>
> RaiMan proposed the following answer:
> see direct communication.
>
> Seems that libs folder was not deleted before repeating the setup.
>
> --
> If this answers your question, please go to the following page to let us
> know that it is solved:
> https://answers.launchpad.net/sikuli/+question/237353/+confirm?answer_id=5
>
> If you still need help, you can reply to this email or go to the
> following page to enter your feedback:
> https://answers.launchpad.net/sikuli/+question/237353
>
> You received this question notification because you asked the question.
>

Revision history for this message
RaiMan (raimund-hocke) said :
#8

-- I can only try the new suggestion in the night
no problem. I have lots of things to do in between too ;-)

-- Windows 8 or Windows 7 should not make any difference

-- I will now check again on my Win7-64 ...
.. and try to run your sample and again check the setup procedure with a fresh empty folder.

-- handling of the libs path on Windows when moving around sikuli stuff
since on Windows I did not find yet a solution to be able to avoid the path entry with the libs folder:
It is true, that Sikuli will try to recreate the libs folder at runtime, preferably in the folder, that contains the Sikuli jar files.
But the libs folder MUST be in the system path at runtime.
So if you have a solution, that uses sikuli-java.jar in different places or even packaged with other stuff on "Sikuli-fresh" systems, you should be aware, that currently on Windows I try to add the path to the registry (but this will only be activated after logout/login), if it is not on system path.
So to avoid this making problems, it is your responsibility to add the correct libs path to the environment before.
The libs handling will further be improved with 1.1

-- Sikuli and Maven
I myself am fully on Maven with the Sikuli development.
For version 1.1+ I plan to have sikuli-java.jar on MavenCentral

Revision history for this message
Best RaiMan (raimund-hocke) said :
#9

Good news - bad news ;-)

good: I understand what is happening ;-)

bad: you cannot use 1.0.1, you have to wait for 1.1 (betas will start next week)

- 1.0.1 does not find your patterns
you are loading the images from Java resource files as BufferedImages and use a Pattern(BufferedImage).
This seem to have worked in 10.0, but apparently was cut with SikuliX, where you only could create a pattern with image files.
To use images from resource files and BufferedImages directly will be available in 1.1

- problem with WinUtil.dll on Windows 8
apparently WinUtil.dll cannot be loaded on your Windows 8 (as your setup log reports)
I will check and try to fix it later towards end of this year, since I have to first get me a Windows 8 installation.

This is the Java code I used to check and confirm:

package sikulix.run101;
import org.sikuli.basics.Debug;
import org.sikuli.script.*;
public class SikuliXRun101 {
  public static void main(String[] args) throws Exception {
    //Debug.setDebugLevel(3);
    Screen s = new Screen();
    ScreenImage img = s.userCapture();
    Pattern pBImg = new Pattern(img.getImage());
    Pattern pFile = new Pattern(img.getFile());
    Debug.user("pattern: %s", pBImg);
    try {
      Debug.user("Match: %s", s.find(pBImg));
    } catch (FindFailed ex) {
      Debug.error(ex.getMessage());
    }
    Debug.user("pattern: %s", pFile);
    try {
      Debug.user("Match: %s", s.find(pFile));
    } catch (FindFailed ex) {
      Debug.error(ex.getMessage());
    }
  }
}

output produced:
[user (16.10.13 13:00:19)] pattern: P(-- BufferedImage --) S: 0.7
[error] can not find P(-- BufferedImage --) S: 0.7 on the screen.
[user (16.10.13 13:00:22)] pattern:
P(/var/folders/wk/pcty7jkx1r5bzc5dvs6n5x_40000gn/T/sikuli-3135788393117307295.png) S: 0.7
[user (16.10.13 13:00:23)] Match: M[71,40 100x33]@S(S(0)[0,0 1920x1200]) S:0,98 Target:121,56

Revision history for this message
RaiMan (raimund-hocke) said :
#10

--- your question about search quality
The search was heavily improved with SikuliX, runs 3 - 5 times faster and and is far more precise.

Revision history for this message
blackheat (iamblackheat) said :
#11

Thanks RaiMan, that solved my question.