Write sikuli code in pure java?

Asked by Stuart Robinson

I would like to include sikuli in a java project I am working on. (namely, a Selenium Webdriver project).

I included the sikuli jar. This is as far as I got (example code followed by resulting exception/stack trace)

****************************************************************
import edu.mit.csail.uid.SikuliScript;
public class Test {
 public static void t1() throws Exception{
  SikuliScript si = new SikuliScript();
  String img = "C:\\Documents and Settings\\Stuart\\My Documents\\sikuliStuff\\test1.sikuli\\1272563063578.png";
  si.doubleClick(img, 0);
 }
 public static void main(String [] args){
  try{ t1(); }catch (Exception e){e.printStackTrace();}
 }
}

Exception in thread "main" java.lang.UnsatisfiedLinkError: C:\Program Files\eclipse\workspace\Selenium_Sikuli\tmplib\ScreenMatchProxy.dll: Can't find dependent libraries
 at java.lang.ClassLoader$NativeLibrary.load(Native Method)
 at java.lang.ClassLoader.loadLibrary0(Unknown Source)
 at java.lang.ClassLoader.loadLibrary(Unknown Source)
 at java.lang.Runtime.load0(Unknown Source)
 at java.lang.System.load(Unknown Source)
 at com.wapmx.nativeutils.jniloader.NativeLoader.loadLibrary(NativeLoader.java:44)
 at edu.mit.csail.uid.ScreenMatchProxy.<clinit>(ScreenMatchProxy.java:11)
 at edu.mit.csail.uid.SikuliScript.<init>(SikuliScript.java:105)
 at Test.t1(Test.java:4)
 at Test.main(Test.java:10)
****************************************************************

any help would be greatly appreciated!

Thanks,
Stuart

Question information

Language:
English Edit question
Status:
Answered
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said :
#1

May be this is of some help, though I think it is for writing something in jython/python.

I got the following information from the developers (I myself did not try this until now ;-)

---------------
(1) The steps to use Sikuli.py without Sikuli IDE:
 1. add sikuli-script.jar into Java's CLASSPATH
 2. add -Dpython.home=sikuli-script.jar as a parameter of launching java
 3. add "from python.edu.mit.csail.uid.Sikuli import *" in your python code. (Note that the package name will be changed in 0.10.)

(2)
Sikuli need some opencv libraries in tmplib/.
Try to add tmplib/ to %PATH% before running Jython.

(3)
In theory, you can import the other xxx.sikuli/xxx.py and call its functions. A tricky step is that you need to call setBundlePath() before using the image files inside the other .sikuli bundle, otherwise Sikuli's functions doesn't know where to find the images.
We will try to make this easier in the future versions.
-----------------

Hope this helps a little down the road. Feedback about your experiences and findings are very much appreciated.

Revision history for this message
Stuart Robinson (stuart-clifford) said :
#2

Hi RaiMan,

Yeah, I saw that in the python/jython post yesterday but I don't know how to apply it to a java setup :-/

Revision history for this message
stuart robinson (strobinso) said :
#3

here is the result of the same test run on mac 10.5

import edu.mit.csail.uid.SikuliScript;
public class Test {
    public static void t1() throws Exception{
        SikuliScript si = new SikuliScript();
        String img = "/Development/eclipseWorkspace/TheWest/bin/1272670092841.png";
        si.doubleClick(img, 0);
    }
    public static void main(String [] args){
        try{ t1(); }catch (Exception e){e.printStackTrace();}
    }
}

Exception in thread "main" java.lang.UnsatisfiedLinkError: /Development/eclipseWorkspace/TheWest/tmplib/libScreenMatchProxy.jnilib: Library not loaded: /opt/local/lib/libcxcore.1.dylib Referenced from: /Development/eclipseWorkspace/TheWest/tmplib/libScreenMatchProxy.jnilib Reason: image not found
    at java.lang.ClassLoader$NativeLibrary.load(Native Method)
    at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1824)
    at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1704)
    at java.lang.Runtime.load0(Runtime.java:769)
    at java.lang.System.load(System.java:968)
    at com.wapmx.nativeutils.jniloader.NativeLoader.loadLibrary(NativeLoader.java:44)
    at edu.mit.csail.uid.ScreenMatchProxy.<clinit>(ScreenMatchProxy.java:11)
    at edu.mit.csail.uid.SikuliScript.<init>(SikuliScript.java:105)
    at Test.t1(Test.java:4)
    at Test.main(Test.java:11)

Revision history for this message
Andrzej Wiech (andrjus-w) said :
#4

I had the same problem but have managed to sort it out, here is how on MAC:

1. Copy Sikuli-IDE.app/Contents/Frameworks folder to the place where you run your java
2. Unpack sikuli-script.jar placed in Sikuli-IDE.app/Contents/Resources/Java and extract libScreenMatchProxy.jnilib file from it
3. Copy libScreenMatchProxy.jnilib to tmplib folder to the place where you run your java

You should now hava Frameworks folder and libScreenMatchProxy.jnilib file in your project

4. Use install_name_tool to change jnilib files (I don't know exactly why, but I know it will works) all from Frameworks folder and
ibScreenMatchProxy.jnilib

e.g.

 otool -L libScreenMatchProxy.jnilib

shows you the path to lib with @exactuable_path variable, you need to change it to @loader_path using following command:

install_name_tool -change "@executable_path/../Frameworks/libtiff.3.dylib" "@loader_path/../Frameworks/libtiff.3.dylib" libScreenMatchProxy.jnilib

The same for all libraries and all paths.

If you experience problems I can send you the right files, just let me know.

The other problem is how to use method like click, dragDrop and other from Java in Sikuli 0.10 as this methods are not part of SikuliScript class anymore but Region class instead I think. Still working on that.

Revision history for this message
Andrzej Wiech (andrjus-w) said :
#5

OK found it:
http://sikuli.org/trac/wiki/revise-api-0.10
It is not Region class but Screen. All works fine :), I am sooo happy.

Andrzej

Revision history for this message
Tsung-Hsiang Chang (vgod) said :
#6

Glad to see you've found how to use them.
It would be great if you can write a blog or an article to share these to Java prople. :)

Revision history for this message
Andrzej Wiech (andrjus-w) said :
#7

With pleasure :)

Wysłane z iPhone'a

Dnia 2010-05-11 o godz. 17:12 Tsung-Hsiang Chang <<email address hidden>
 > napisał(a):

> Question #108878 on Sikuli changed:
> https://answers.launchpad.net/sikuli/+question/108878
>
> Tsung-Hsiang Chang posted a new comment:
> Glad to see you've found how to use them.
> It would be great if you can write a blog or an article to share
> these to Java prople. :)
>
> --
> You received this question notification because you are a direct
> subscriber of the question.

Revision history for this message
Libo Cao (libocao) said :
#8

In version 0.10 Sikuli,

The libScreenMatchProxy.jnilib extracted from
/Applications/Sikuli-IDE.app/Contents/Resources/Java/sikuli-script.jar

is different from
/Applications/Sikuli-IDE.app/Contents/Frameworks/libScreenMatchProxy.jnilib

Why is that?

Revision history for this message
Libo Cao (libocao) said :
#9

In version 0.10 Sikuli,

The libScreenMatchProxy.jnilib extracted from
/Applications/Sikuli-IDE.app/Contents/Resources/Java/sikuli-script.jar

is different from
/Applications/Sikuli-IDE.app/Contents/Frameworks/libScreenMatchProxy.jnilib

Why is that?

Revision history for this message
Libo Cao (libocao) said :
#10

In version 0.10 Sikuli,

The libScreenMatchProxy.jnilib extracted from
/Applications/Sikuli-IDE.app/Contents/Resources/Java/sikuli-script.jar

is different from
/Applications/Sikuli-IDE.app/Contents/Frameworks/libScreenMatchProxy.jnilib

Why is that?

Revision history for this message
Tsung-Hsiang Chang (vgod) said :
#11

The only difference is the paths to its dependent dylibs. You can see it using otool -L libScreenMatchProxy.jnilib.
If you would like to use sikuli-script.jar independently, you have to have opencv installed at /opt/local (the default path of Darwin Ports). Otherwise, you need to change the paths to Sikuli-IDE.app/Contents/Frameworks using install_name_tool.

Revision history for this message
Libo Cao (libocao) said :
#12

The library they use are different too:

the one inside Contents/Frameworks needs the following stuff

../../../target/lib/libScreenMatchProxy.jnilib (compatibility version 0.0.0, current version 0.0.0)
 /opt/local/lib/libcv.4.dylib (compatibility version 5.0.0, current version 5.0.0)
 /opt/local/lib/libhighgui.4.dylib (compatibility version 5.0.0, current version 5.0.0)
 /opt/local/lib/libcvaux.4.dylib (compatibility version 5.0.0, current version 5.0.0)
 /opt/local/lib/libml.4.dylib (compatibility version 5.0.0, current version 5.0.0)
 /opt/local/lib/libcxcore.4.dylib (compatibility version 5.0.0, current version 5.0.0)
 /System/Library/Frameworks/JavaVM.framework/Versions/A/JavaVM (compatibility version 1.0.0, current version 1.0.0)
 /opt/local/lib/libtiff.3.dylib (compatibility version 13.0.0, current version 13.2.0)
 /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.9.0)
 /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.0.1)

The one extracted needs the following:
 ../../../target/classes/libScreenMatchProxy.jnilib (compatibility version 0.0.0, current version 0.0.0)
 /opt/local/lib/libcxcore.1.dylib (compatibility version 2.0.0, current version 2.0.0)
 /opt/local/lib/libcv.1.dylib (compatibility version 2.0.0, current version 2.0.0)
 /opt/local/lib/libhighgui.1.dylib (compatibility version 2.0.0, current version 2.0.0)
 /opt/local/lib/libcvaux.1.dylib (compatibility version 2.0.0, current version 2.0.0)
 /opt/local/lib/libml.1.dylib (compatibility version 2.0.0, current version 2.0.0)
 /System/Library/Frameworks/JavaVM.framework/Versions/A/JavaVM (compatibility version 1.0.0, current version 1.0.0)
 /opt/local/lib/libtiff.3.dylib (compatibility version 12.0.0, current version 12.2.0)
 /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.9.0)
 /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 103.0.0)
 /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.0.0)

I am trying to figure out Andrzej Wiech's method and wonder why it was using the extracted ones. Looking forward to his detailed blog or post to get command line working on mac.

Revision history for this message
Libo Cao (libocao) said :
#13

After 5 hours of trying and it is still not working for Sikuli 1.0 command line java mode

Followed instruction from Andrzej above. (assume my home dir is Test)

1. Copy Sikuli-IDE.app/Contents/Frameworks folder to Test folder
2. Unpack sikuli-script.jar placed in Sikuli-IDE.app/Contents/Resources/Java and extract libScreenMatchProxy.jnilib file from it
3. Copy libScreenMatchProxy.jnilib to tmplib folder and put it under Test dir
4. Use install_name_tool to change jnilib files from Frameworks folder - there are only two applied the libScreenMatchProxy.jnilib and the libVDictProxy.jnilib
first use otool to find all that needs to be changed
e.g.

 otool -L libScreenMatchProxy.jnilib

and did the following for all the two libraries (libScreenMatchProxy.jnilib and the libVDictProxy.jnilib) with all the dylib paths:
cd Test/Frameworks ;
install_name_tool -change "@executable_path/../Frameworks/libtiff.3.dylib" "@loader_path/../Frameworks/libtiff.3.dylib" libScreenMatchProxy.jnilib

5) Since I need a Lib dir, so also extract the Lib dir from sikuli-script.jar and put it under Test dir

6) Start execute my script under Test folder, for example
java -d32 -Dpython.path=Lib/ -jar sikuli-script.jar ..//TestSuites/test.sikuli

Still get library loading issues:

ScreenMatchProxy loaded.
Exception in thread "main" Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "Lib/sikuli/Sikuli.py", line 28, in <module>
    from VDict import *
  File "Lib/sikuli/VDict.py", line 1, in <module>
    from edu.mit.csail.uid import VDictProxy
java.lang.UnsatisfiedLinkError: /Users/liboc/Test/tmplib/libVDictProxy.jnilib: Library not loaded: @executable_path/../Frameworks/libcxcore.4.dylib Referenced from: /opt/local/lib/libcv.4.dylib Reason: image not found

Also my confusion is that the jnilib file extracted from sikuli-script.jar is totally different from the one inside Frameworks, I tried both to put inside tmplib file, but always have different library issues, even though I already executed step 4

Revision history for this message
Tsung-Hsiang Chang (vgod) said :
#14

The correct libScreenMatchProxy.jnilib is in sikuli-script.jar/META-INF/lib. The one that depends on /opt/local/lib/libcxcore.1.dylib is an old lib, which should not be packaged into the jar.

Revision history for this message
Tsung-Hsiang Chang (vgod) said :
#15

The step you missed is to change the reference paths in *.dylib using install_name_tool as well.
The following error message tells you what you need to change.
java.lang.UnsatisfiedLinkError: /Users/liboc/Test/tmplib/libVDictProxy.jnilib: Library not loaded: @executable_path/../Frameworks/libcxcore.4.dylib Referenced from: /opt/local/lib/libcv.4.dylib Reason: image not found

Revision history for this message
Libo Cao (libocao) said :
#16

Ok, so you are saying I should do the following 2

1) In my step 2, I should copy libScreenMatchProxy.jnilib is in sikuli-script.jar/META-INF/lib and put it into tmplib, this means Andrzej Wiech's method above shouldn't have worked at all, since he was using the wrong libScreenMatchProxy.jnilib

2) In my Frameworks folder, not only change the *.jnilib file reference path, but also all the *.dylib path.

Will try that and see what happens.

Revision history for this message
Libo Cao (libocao) said :
#17

It still won't work, my question is why do I need to change all the reference path inside the Frameworks folder, but when I run a script, all it is using is tmplib dir libraries?

I try to change the reference path for tmplib/libScreenMatchProxy.jnilib under tmplib folder, but not sure how to change it, since it is all referencing to /opt/local/lib directly by doing something like this

install_name_tool -change "/opt/local/lib/libhighgui.4.dylib" "@loader_path/../Frameworks/libhighgui.4.dylib" libScreenMatchProxy.jnilib

Still not working when execute under Test folder, the folder structure is
Test
     test.sikuli
     Frameworks
     Lib -- also extracted from sikuli-script.jar
     sikuli-script.jar
     tmplib
          /libScreenMatchProxy.jnilib

Execution is under Test dir as

java -d32 -Dpython.path=Lib/ -jar sikuli-script.jar test.sikuli

Error message:

java.lang.UnsatisfiedLinkError: java.lang.UnsatisfiedLinkError: Test/tmplib/libScreenMatchProxy.jnilib: Library not loaded: @executable_path/../Frameworks/libcxcore.4.dylib Referenced from: Test/tmplib/../Frameworks/libhighgui.4.dylib Reason: image not found

Revision history for this message
Tsung-Hsiang Chang (vgod) said :
#18

It seems you didn't change all reference paths in Frameworks/*.dylib.

Try running this script in Frameworks/. (modified from Frameworks/change-lib-path.sh)

---
#!/bin/sh
for dylib in *.dylib
do
   for ref in `otool -L $dylib | grep executable_path | awk '{print $1'}`
   do
      install_name_tool -change $ref @loader_path/../Frameworks/`basename $ref` $dylib
   done
done

---
After this, use otool -L *.dylib to make sure all @executable_path are replaced by @loader_path.

Revision history for this message
Libo Cao (libocao) said :
#19

No that won't work, I checked everything under Frameworks and all path are already changed.

If you look at my error message above, it was complaining it cannot load correctly from tmplib/libScreenMatchProxy.jnilib , which is totally different from the one inside Frameworks that got changed path. the libScreenMatchProxy.jnilib was copied from the sikuli-script.jar//META-INF/lib

if you do otool -L on the tmplib/libScreenMatchProxy.jnilib they are all pointing to directly to /opt/local/lib/*.dylib, there is no @executable_path to replace, I wasn't sure what to do there to make the library link there.

This is also the part confused me that why the library got used is under tmplib, but everyone is telling me keep changing the path inside Frameworks. They are totally different.

Revision history for this message
Andrzej Wiech (andrjus-w) said :
#20

Hi libo

I checked, and it seems that I deleted libScreenMatchProxy.jnilib file from /Frameworks folder. I didn't know if would be problematic if it's there, just though it is there by mistake and removed it. Have you managed to make it work?

BR,
Andrzej

Revision history for this message
Tsung-Hsiang Chang (vgod) said :
#21

@libo,

OK. Let me explain this.
If you are running Sikuli-IDE.app, Sikuli looks for the jnilibs in Frameworks, which depend on some dylibs that are also in Frameworks. So, if you do otool -L *.dylib *.jnilib in Frameworks, you can see all reference paths are pointed to @executable_path/../Frameworks/.

But if you run sikuli-script.jar as a standalone jar, things are different. sikuli-script.jar can extract the jnilibs to tmplib/ automatically, but these jnilibs depend on /opt/local/lib/*.dylib. So, if you don't have opencv installed in /opt/local/lib, you have to change these path in the jnilibs as well. This was the step you missed.
You can use the original Frameworks/change-lib-path.sh with a little modification to change the paths in tmplib/*.jnilib. The original script replaces all /opt/local/bin with @executable_path/../Frameworks. You need to change @executable_path to @loader_path, and that's it.

If everything is good, you should see all reference paths start with @loader_path in tmplib/*.jnilib and Frameworks/*.dylib.

Revision history for this message
Libo Cao (libocao) said :
#22

Thanks Chang and Andrzej for your patience. Still not working ...

@Andrzej: deleting the libScreenMatchProxy.jnilib file under Frameworks won't help, since that one was never get used when I am using method 2 (it uses the tmplib, which Chang said should be extracted from META-INF/lib instead of the root one under sikuli-script.jar as you mentioned in your original post, he said that one was packed by mistake.)

@Chang:
I was using method 2 from the two methods Chang mentioned above. And let's sort out what happened

1) To generate the tmplib, I followed your instruction using the sikuli-script.jar extracted libScreenMatchProxy.jnilib from META-INF/lib
2) Changed all the path for libScreenMatchProxy.jnilib and now it all looks like this

eng14:Sikuli-Mac-Lib_1.0 $ cd tmplib
eng14:tmplib amitp$ otool -L libScreenMatchProxy.jnilib
libScreenMatchProxy.jnilib:
 ../../../target/lib/libScreenMatchProxy.jnilib (compatibility version 0.0.0, current version 0.0.0)
 @loader_path/../Frameworks/libcv.4.dylib (compatibility version 5.0.0, current version 5.0.0)
 @loader_path/../Frameworks/libhighgui.4.dylib (compatibility version 5.0.0, current version 5.0.0)
 @loader_path/../Frameworks/libcvaux.4.dylib (compatibility version 5.0.0, current version 5.0.0)
 /opt/local/lib/libml.4.dylib (compatibility version 5.0.0, current version 5.0.0)
 @loader_path/../Frameworks/libcxcore.4.dylib (compatibility version 5.0.0, current version 5.0.0)
 /System/Library/Frameworks/JavaVM.framework/Versions/A/JavaVM (compatibility version 1.0.0, current version 1.0.0)
 @loader_path/../Frameworks/libtiff.3.dylib (compatibility version 13.0.0, current version 13.2.0)
 /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.9.0)
 /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.0.1)

3) I checked and I do have all those libraries inside my ../Frameworks folder

4) execute script that is at ../tmplib folder like this

java -d32 -Dpython.path=Lib/ -jar sikuli-script.jar test.sikuli

Error message:

java.lang.UnsatisfiedLinkError: java.lang.UnsatisfiedLinkError: Test/tmplib/libScreenMatchProxy.jnilib: Library not loaded: @executable_path/../Frameworks/libcxcore.4.dylib Referenced from: Test/tmplib/../Frameworks/libhighgui.4.dylib Reason: image not found

Revision history for this message
Libo Cao (libocao) said :
#23

Also method 1 won't work neither for example:

open -a /Applications/Sikuli-IDE.app /Users/liboc/client_automation/iphoneATT/TestSuites/POI/POI.skl

Will give you run time error from Sikuli IDE, it was complaining an imported library from POI.sikuli test suite was using lines like "import sikuli.Sikuli.*"

Also I don't think this method is scalable, since it is hard to catch the popup error message when you are running things in a batch.

Note that I had hundreds of testcases inside tens of testsuites running fine with the method 2 before upgrading to Sikuli 1.0, however I cannot get the tmplib working now.

I wonder if anyone got Sikuli 1.0 working in a command line mode by calling in the way that was working for older version of Sikuli:

java -d32 -Dpython.path=Lib/ -jar sikuli-script.jar test.sikuli

Note that the Lib was extracted from sikuli-script.jar and I was told the tmplib should have some lib and path needs to be fixed somehow to get it working. In the older version, the tmplib was automatically generated.

I got this method working for older version of Sikuli from this post:

https://answers.launchpad.net/sikuli/+question/100436

Revision history for this message
Tsung-Hsiang Chang (vgod) said :
#24

Try this package: http://people.csail.mit.edu/vgod/sikuli/sikuli-libs.zip
It should have all correct dylibs and jnilibs you need.

Revision history for this message
Andrzej Wiech (andrjus-w) said :
#25

I got it working libo. I will write some guide on it soon, meanwhile
if you want I can send u packed example for Python and Java.

Andrzej

Wysłane z iPhone'a

Dnia 2010-05-14 o godz. 21:16 libo
<email address hidden> napisał(a):

> Question #108878 on Sikuli changed:
> https://answers.launchpad.net/sikuli/+question/108878
>
> libo posted a new comment:
> Also method 1 won't work neither for example:
>
> open -a /Applications/Sikuli-IDE.app
> /Users/liboc/client_automation/iphoneATT/TestSuites/POI/POI.skl
>
> Will give you run time error from Sikuli IDE, it was complaining an
> imported library from POI.sikuli test suite was using lines like
> "import
> sikuli.Sikuli.*"
>
> Also I don't think this method is scalable, since it is hard to catch
> the popup error message when you are running things in a batch.
>
> Note that I had hundreds of testcases inside tens of testsuites
> running
> fine with the method 2 before upgrading to Sikuli 1.0, however I
> cannot
> get the tmplib working now.
>
> I wonder if anyone got Sikuli 1.0 working in a command line mode by
> calling in the way that was working for older version of Sikuli:
>
> java -d32 -Dpython.path=Lib/ -jar sikuli-script.jar test.sikuli
>
> Note that the Lib was extracted from sikuli-script.jar and I was told
> the tmplib should have some lib and path needs to be fixed somehow to
> get it working. In the older version, the tmplib was automatically
> generated.
>
> I got this method working for older version of Sikuli from this post:
>
> https://answers.launchpad.net/sikuli/+question/100436
>
> --
> You received this question notification because you are a direct
> subscriber of the question.

Revision history for this message
Libo Cao (libocao) said :
#26

Wow, magical, all those libraries loaded now, it is only complaining that import sikuli.Sikuli.* is not valid, for older versions it works fine as "from python.edu.mit.csail.uid.Sikuli import *

eng14:Sikuli-Mac-Lib_1.0$ ./run_all_tests.sh
ScreenMatchProxy loaded.
VDictProxy loaded.
ScreenMatchProxy loaded.
Exception in thread "main" Traceback (most recent call last):
  File "/Users/liboc/CATS/trunk/client_automation/Sikuli-Mac-Lib_1.0/../iphoneATT/TestSuites/POI/POI.sikuli/POI.py", line 5, in <module>
    import search
SyntaxError: ("mismatched input '*' expecting set null", ('/Users/liboc/CATS/trunk/client_automation/iphoneATT/Lib/search.py', 3, 21, 'import sikuli.Sikuli.*\n'))

Revision history for this message
Libo Cao (libocao) said :
#27

Thanks Andrzej, Chang's files also helps loading all the libraries. I look forward to your detailed guide, since I have no idea where Chang got those files under tmplib, since in the original post it only mentioned one library, but there are three in his tmplib.

Now it goes back to the question of the original post
The older version to import sikuli works fine
from python.edu.mit.csail.uid.Sikuli import *

but not the new one

import sikuli.Sikuli.* is not valid to me.

In my test suite, I was calling another python library that implement all the routines, in which it has to import sikuli somehow.

I would also write a post on how to implement large scale testing framework using latest Sikuli 1.0 after I resolve this issue.

Revision history for this message
Andrzej Wiech (andrjus-w) said :
#28

I think from edu.mit.csail.uid.SikuliScript import * should do. It would be very interesting to see how you dealt with large scale testing. I actually run Sikuli from java and use jUnit as testing framework together with maven. I switch from Python to Java and so far I am glad, maybe cause I am more familiar with Java then with Java. Actually I just checked,I have that import commented and all Sikuli method like click or find are visible in my python test anyway.

Revision history for this message
Tsung-Hsiang Chang (vgod) said :
#29

@libo:

The three jnilibs are copied from Sikuli-IDE.app//Contents/Frameworks/. I just changed the paths of dependent libs in them. They are identical with the ones in META-INF/lib.

Can you see if you have sikuli/ under Lib/? If not, you may extract the Lib from a wrong sikuli-script.jar.

Revision history for this message
Libo Cao (libocao) said :
#30

@Andrzej,

I changed the import to from edu.mit.csail.uid.SikuliScript import * , looks like the test is running, but it does not even recognize find(), so I am not sure if that is the right path to import, or I should dig into Sikuli 1.0 documentation to see if find actually changed into something else.

@Chang,

There is a sikuli under Lib, and I got that Lib extracted from the sikuli-script.jar under
/Applications/Sikuli-IDE.app/Contents/Resources/Java/sikuli-script.jar

Hopefully that is the one I am suppose to use.

Revision history for this message
Libo Cao (libocao) said :
#31

@Andrzej and @Chang,

I finally also got the import working from other python script, you need to use

from sikuli.Sikuli import *

instead of

from edu.mit.csail.uid.SikuliScript import *
or
import sikuli.Sikuli.*

Now start the hard work to convert all my old find() to new way of using them -- define object of Region or Screen first and call it within the object.

Thanks guys,

I promise to write a post on this migration experience and how to build large scaled testing framework using Sikuli

Libo

Revision history for this message
Tsung-Hsiang Chang (vgod) said :
#32

@libo,

Congrats on figuring out the problem. :)
Looking forward to your post.

Revision history for this message
Libo Cao (libocao) said :
#33

One more thing though.

In sikuli 1.0 documentation, it mentioned that people can call find(), click() etc directly and by default sikuli will assume it is calling the one in Screen class. This works inside the IDE

However when you try to use it from a python script by importing
import sikuli.Sikuli.*
It only recognize those methods inside sikuli/Sikuli.py as global methods. When you try to use find() directly, it won't allow you. Why is that? Shouldn't both method works based on documentation?

Maybe I am missing something here. note that edu.mit.csail.uid is not packed inside Lib any more, they are now only java classes outside the Lib folder, so nothing else for me to include.

Revision history for this message
Tsung-Hsiang Chang (vgod) said :
#34

edu.mit.csail.uid is a Java package, not for Jython. If you are using Jython scripts, you should use "from sikuli.Sikuli import *".
The global functions are constructed automatically when you import sikuli.Sikuli. It should work even without the IDE. (See the function initSikuli() in sikuli/Sikuli.py if you are interested in the details.)

Revision history for this message
Libo Cao (libocao) said :
#35

@Chang,

I read the source, and every python script that include the line of

from sikuli.Sikuli import *

should be able to call find() and click() etc.

I AM able to do that if importing directly from a sikuli script. However this is what not working:

If you write a sikuli script, and inside your test.py file, you import another python Class/module, and inside that class you try to use click() or find(), you won't be able to. Even if you imported sikuli inside that class too.

Revision history for this message
Libo Cao (libocao) said :
#36

Here is one example:

1) Make a file test.py inside test.sikuli
import sys
import unittest
import recorder
from sikuli.Sikuli import *

class Test(unittest.TestCase):
    def setUp(self):
        setThrowException(False)

    def tearDown(self):
        setThrowException(True)

    def test_direct_call(self):
        a = find("PATH_TO_AN_IMAGE")
        if a:
            click(a)

    def test_indirect_call(self):
        recorder_obj = recorder.Recorder()
        recorder_obj.find_and_click()

suite = unittest.TestLoader().loadTestsFromTestCase(Test)
unittest.TextTestRunner(verbosity=2).run(suite)

2) Make a class file named recorder.py

from sikuli.Sikuli import *

class Recorder:
    def __init__(self):
        pass

    def find_and_click(self):
        a = find("PATH_TO_AN_IMAGE")
        if a:
            click(a)

3) make sure you replace the PATH_TO_AN_IMAGE to any image locally
4) At the folder with all the setup folder (the files you sent to me) tmplib, Lib, sikuli-script.jar, Frameworks etc, run the following command:

java -d32 -Dpython.path=Lib/ -jar sikuli-script.jar test.sikuli

And as a result, you should see your first test passed, but the second test will fail with exception telling you "find" name error.

Revision history for this message
Tsung-Hsiang Chang (vgod) said :
#37

I see.
It's a name space problem.
The initSikuli() function only exposes the functions to __main__. So, if you import sikuli from a module other than __main__, you won't be able to see the Region's methods as global functions.

It's a little tricky here. I need some time to think about an elegant way to fix this for modules.

Revision history for this message
Libo Cao (libocao) said :
#38

Thanks Chang, hopefully we can figure out a way to resolve this.

When I was using older version Sikuli, it worked out fine though, I haven't check the source on that version of Sikuli yet.

To build a large scale system, usually you won't write your methods inside testsuites, instead you write them inside a separate library (then many test cases and testsuites can use the same library methods) so this problem must be resolved to make Sikuli more suitable for larger systems.

In the mean while if we cannot resolve this, I will roll back to older version of Sikuli, which supports this feature

Revision history for this message
Tsung-Hsiang Chang (vgod) said :
#39

@libo,

I have figured out a way to resolve this. :)
Here is the patch, http://people.csail.mit.edu/vgod/sikuli/sikuli-mod.patch, if you would like to test if it works.

Revision history for this message
Libo Cao (libocao) said :
#40

How do you apply the patch?
------Original Message------
From: Tsung-Hsiang Chang
Sender: <email address hidden>
To: <email address hidden>
ReplyTo: <email address hidden>
Subject: Re: [Question #108878]: Write sikuli code in pure java?
Sent: May 15, 2010 9:05 AM

Question #108878 on Sikuli changed:
https://answers.launchpad.net/sikuli/+question/108878

Tsung-Hsiang Chang posted a new comment:
@libo,

I have figured out a way to resolve this. :)
Here is the patch, http://people.csail.mit.edu/vgod/sikuli/sikuli-mod.patch, if you would like to test if it works.

--
You received this question notification because you are a direct
subscriber of the question.

Sent via BlackBerry by AT&T

Revision history for this message
Libo Cao (libocao) said :
#41

You know since I am running command line mode on mac without IDE and we did a bunch of file copy and path changing stuff. I have Lib, Frameworks and tmplib under testing dir
------Original Message------
From: libo
Sender: <email address hidden>
To: <email address hidden>
ReplyTo: <email address hidden>
Subject: Re: [Question #108878]: Write sikuli code in pure java?
Sent: May 15, 2010 11:03 AM

Question #108878 on Sikuli changed:
https://answers.launchpad.net/sikuli/+question/108878

libo proposed the following answer:
How do you apply the patch?
------Original Message------
From: Tsung-Hsiang Chang
Sender: <email address hidden>
To: <email address hidden>
ReplyTo: <email address hidden>
Subject: Re: [Question #108878]: Write sikuli code in pure java?
Sent: May 15, 2010 9:05 AM

Question #108878 on Sikuli changed:
https://answers.launchpad.net/sikuli/+question/108878

Tsung-Hsiang Chang posted a new comment:
@libo,

I have figured out a way to resolve this. :)
Here is the patch, http://people.csail.mit.edu/vgod/sikuli/sikuli-mod.patch, if you would like to test if it works.

--
You received this question notification because you are a direct
subscriber of the question.

Sent via BlackBerry by AT&T

You received this question notification because you are a direct
subscriber of the question.

Sent via BlackBerry by AT&T

Revision history for this message
Libo Cao (libocao) said :
#42

Hi Chang,

I know you and Raiman do not lack of oppertunities at all, but I have to ask. If you guys do ever need summer internship or a full time job, please let me know. I work for Telenav, which just went IPO last thursday(TNAV) and it is a fast growing company, we have server side, client side, and auto embeded applications, there are tons of oppertunities here, just let me know if you guys would be interested, it would be exciting to work with you guys
Libo
Sent via BlackBerry by AT&T

Revision history for this message
Libo Cao (libocao) said :
#43

My appologies, the above message is meant to send only to Chang, I was using my blackberry to reply to you name only but it ended up here, please delete it if possible
------Original Message------
From: libo
Sender: <email address hidden>
To: <email address hidden>
ReplyTo: <email address hidden>
Subject: [Question #108878]: Do you guys need any summer internship or a fulltime job ?
Sent: May 15, 2010 11:21 AM

Question #108878 on Sikuli changed:
https://answers.launchpad.net/sikuli/+question/108878

libo proposed the following answer:
Hi Chang,

I know you and Raiman do not lack of oppertunities at all, but I have to ask. If you guys do ever need summer internship or a full time job, please let me know. I work for Telenav, which just went IPO last thursday(TNAV) and it is a fast growing company, we have server side, client side, and auto embeded applications, there are tons of oppertunities here, just let me know if you guys would be interested, it would be exciting to work with you guys
Libo
Sent via BlackBerry by AT&T

--
You received this question notification because you are a direct
subscriber of the question.

Sent via BlackBerry by AT&T

Revision history for this message
Tsung-Hsiang Chang (vgod) said :
#44

@libo,

It seems there is no way to delete messages here...
Thanks for your kindness. I already have an internship this summer.
But maybe we will have opportunities to collaborate together. :)

To answer your question of how to apply the patch, just run "patch < sikuli-mod.patch" under Lib/sikuli.

Revision history for this message
Libo Cao (libocao) said :
#45

Awesome! the patch works, now it can recognize methods by import sikuli.Sikuli.* from imported modules.

Only note is that when the patch is asking you which file to patch, if you are running under Lib dir, you need to give your full path to current dir then add /sikuli/Screen.py and /sikuli/Sikuli.py

Thanks to Chang and Andrzej for helping me for this migration, I will make a new post on this thread to share the procedures on running Sikuli 1.0 under command line mode for large scale testing system.

Revision history for this message
Tsung-Hsiang Chang (vgod) said :
#46

@libo,

Great! Looking forward to your post. :)

Revision history for this message
Libo Cao (libocao) said :
#47

Posted
"Building Large-scale Testing framework Using Sikuli 1.0 - Mac example "

https://answers.launchpad.net/sikuli/+question/111193/+index

Thanks to Chang and Andrzej for your help with this upgrade.

Revision history for this message
Tsung-Hsiang Chang (vgod) said :
#48

Thanks, libo!

Can you help with this problem?

Provide an answer of your own, or ask Stuart Robinson for more information if necessary.

To post a message you must log in.