Write sikuli code in pure java?
I would like to include sikuli in a java project I am working on. (namely, a Selenium Webdriver project).
I included the sikuli jar. This is as far as I got (example code followed by resulting exception/stack trace)
*******
import edu.mit.
public class Test {
public static void t1() throws Exception{
SikuliScript si = new SikuliScript();
String img = "C:\\Documents and Settings\
si.doubleClic
}
public static void main(String [] args){
try{ t1(); }catch (Exception e){e.printStack
}
}
Exception in thread "main" java.lang.
at java.lang.
at java.lang.
at java.lang.
at java.lang.
at java.lang.
at com.wapmx.
at edu.mit.
at edu.mit.
at Test.t1(
at Test.main(
*******
any help would be greatly appreciated!
Thanks,
Stuart
Question information
- Language:
- English Edit question
- Status:
- Answered
- For:
- SikuliX Edit question
- Assignee:
- No assignee Edit question
- Last query:
- Last reply:
Revision history for this message
|
#1 |
May be this is of some help, though I think it is for writing something in jython/python.
I got the following information from the developers (I myself did not try this until now ;-)
---------------
(1) The steps to use Sikuli.py without Sikuli IDE:
1. add sikuli-script.jar into Java's CLASSPATH
2. add -Dpython.
3. add "from python.
(2)
Sikuli need some opencv libraries in tmplib/.
Try to add tmplib/ to %PATH% before running Jython.
(3)
In theory, you can import the other xxx.sikuli/xxx.py and call its functions. A tricky step is that you need to call setBundlePath() before using the image files inside the other .sikuli bundle, otherwise Sikuli's functions doesn't know where to find the images.
We will try to make this easier in the future versions.
-----------------
Hope this helps a little down the road. Feedback about your experiences and findings are very much appreciated.
Revision history for this message
|
#2 |
Hi RaiMan,
Yeah, I saw that in the python/jython post yesterday but I don't know how to apply it to a java setup :-/
Revision history for this message
|
#3 |
here is the result of the same test run on mac 10.5
import edu.mit.
public class Test {
public static void t1() throws Exception{
String img = "/Development/
}
public static void main(String [] args){
try{ t1(); }catch (Exception e){e.printStack
}
}
Exception in thread "main" java.lang.
at java.lang.
at java.lang.
at java.lang.
at java.lang.
at java.lang.
at com.wapmx.
at edu.mit.
at edu.mit.
at Test.t1(
at Test.main(
Revision history for this message
|
#4 |
I had the same problem but have managed to sort it out, here is how on MAC:
1. Copy Sikuli-
2. Unpack sikuli-script.jar placed in Sikuli-
3. Copy libScreenMatchP
You should now hava Frameworks folder and libScreenMatchP
4. Use install_name_tool to change jnilib files (I don't know exactly why, but I know it will works) all from Frameworks folder and
ibScreenMatchPr
e.g.
otool -L libScreenMatchP
shows you the path to lib with @exactuable_path variable, you need to change it to @loader_path using following command:
install_name_tool -change "@executable_
The same for all libraries and all paths.
If you experience problems I can send you the right files, just let me know.
The other problem is how to use method like click, dragDrop and other from Java in Sikuli 0.10 as this methods are not part of SikuliScript class anymore but Region class instead I think. Still working on that.
Revision history for this message
|
#5 |
OK found it:
http://
It is not Region class but Screen. All works fine :), I am sooo happy.
Andrzej
Revision history for this message
|
#6 |
Glad to see you've found how to use them.
It would be great if you can write a blog or an article to share these to Java prople. :)
Revision history for this message
|
#7 |
With pleasure :)
Wysłane z iPhone'a
Dnia 2010-05-11 o godz. 17:12 Tsung-Hsiang Chang <<email address hidden>
> napisał(a):
> Question #108878 on Sikuli changed:
> https:/
>
> Tsung-Hsiang Chang posted a new comment:
> Glad to see you've found how to use them.
> It would be great if you can write a blog or an article to share
> these to Java prople. :)
>
> --
> You received this question notification because you are a direct
> subscriber of the question.
Revision history for this message
|
#8 |
In version 0.10 Sikuli,
The libScreenMatchP
/Applications/
is different from
/Applications/
Why is that?
Revision history for this message
|
#9 |
In version 0.10 Sikuli,
The libScreenMatchP
/Applications/
is different from
/Applications/
Why is that?
Revision history for this message
|
#10 |
In version 0.10 Sikuli,
The libScreenMatchP
/Applications/
is different from
/Applications/
Why is that?
Revision history for this message
|
#11 |
The only difference is the paths to its dependent dylibs. You can see it using otool -L libScreenMatchP
If you would like to use sikuli-script.jar independently, you have to have opencv installed at /opt/local (the default path of Darwin Ports). Otherwise, you need to change the paths to Sikuli-
Revision history for this message
|
#12 |
The library they use are different too:
the one inside Contents/Frameworks needs the following stuff
../../.
/opt/local/
/opt/local/
/opt/local/
/opt/local/
/opt/local/
/System/
/opt/local/
/usr/lib/
/usr/lib/
The one extracted needs the following:
../../
/opt/local/
/opt/local/
/opt/local/
/opt/local/
/opt/local/
/System/
/opt/local/
/usr/lib/
/usr/lib/
/usr/lib/
I am trying to figure out Andrzej Wiech's method and wonder why it was using the extracted ones. Looking forward to his detailed blog or post to get command line working on mac.
Revision history for this message
|
#13 |
After 5 hours of trying and it is still not working for Sikuli 1.0 command line java mode
Followed instruction from Andrzej above. (assume my home dir is Test)
1. Copy Sikuli-
2. Unpack sikuli-script.jar placed in Sikuli-
3. Copy libScreenMatchP
4. Use install_name_tool to change jnilib files from Frameworks folder - there are only two applied the libScreenMatchP
first use otool to find all that needs to be changed
e.g.
otool -L libScreenMatchP
and did the following for all the two libraries (libScreenMatch
cd Test/Frameworks ;
install_name_tool -change "@executable_
5) Since I need a Lib dir, so also extract the Lib dir from sikuli-script.jar and put it under Test dir
6) Start execute my script under Test folder, for example
java -d32 -Dpython.path=Lib/ -jar sikuli-script.jar ..//TestSuites/
Still get library loading issues:
ScreenMatchProxy loaded.
Exception in thread "main" Traceback (most recent call last):
File "<string>", line 1, in <module>
File "Lib/sikuli/
from VDict import *
File "Lib/sikuli/
from edu.mit.csail.uid import VDictProxy
java.lang.
Also my confusion is that the jnilib file extracted from sikuli-script.jar is totally different from the one inside Frameworks, I tried both to put inside tmplib file, but always have different library issues, even though I already executed step 4
Revision history for this message
|
#14 |
The correct libScreenMatchP
Revision history for this message
|
#15 |
The step you missed is to change the reference paths in *.dylib using install_name_tool as well.
The following error message tells you what you need to change.
java.lang.
Revision history for this message
|
#16 |
Ok, so you are saying I should do the following 2
1) In my step 2, I should copy libScreenMatchP
2) In my Frameworks folder, not only change the *.jnilib file reference path, but also all the *.dylib path.
Will try that and see what happens.
Revision history for this message
|
#17 |
It still won't work, my question is why do I need to change all the reference path inside the Frameworks folder, but when I run a script, all it is using is tmplib dir libraries?
I try to change the reference path for tmplib/
install_name_tool -change "/opt/local/
Still not working when execute under Test folder, the folder structure is
Test
test.sikuli
Frameworks
Lib -- also extracted from sikuli-script.jar
sikuli-
tmplib
Execution is under Test dir as
java -d32 -Dpython.path=Lib/ -jar sikuli-script.jar test.sikuli
Error message:
java.lang.
Revision history for this message
|
#18 |
It seems you didn't change all reference paths in Frameworks/*.dylib.
Try running this script in Frameworks/. (modified from Frameworks/
---
#!/bin/sh
for dylib in *.dylib
do
for ref in `otool -L $dylib | grep executable_path | awk '{print $1'}`
do
install_
done
done
---
After this, use otool -L *.dylib to make sure all @executable_path are replaced by @loader_path.
Revision history for this message
|
#19 |
No that won't work, I checked everything under Frameworks and all path are already changed.
If you look at my error message above, it was complaining it cannot load correctly from tmplib/
if you do otool -L on the tmplib/
This is also the part confused me that why the library got used is under tmplib, but everyone is telling me keep changing the path inside Frameworks. They are totally different.
Revision history for this message
|
#20 |
Hi libo
I checked, and it seems that I deleted libScreenMatchP
BR,
Andrzej
Revision history for this message
|
#21 |
@libo,
OK. Let me explain this.
If you are running Sikuli-IDE.app, Sikuli looks for the jnilibs in Frameworks, which depend on some dylibs that are also in Frameworks. So, if you do otool -L *.dylib *.jnilib in Frameworks, you can see all reference paths are pointed to @executable_
But if you run sikuli-script.jar as a standalone jar, things are different. sikuli-script.jar can extract the jnilibs to tmplib/ automatically, but these jnilibs depend on /opt/local/
You can use the original Frameworks/
If everything is good, you should see all reference paths start with @loader_path in tmplib/*.jnilib and Frameworks/*.dylib.
Revision history for this message
|
#22 |
Thanks Chang and Andrzej for your patience. Still not working ...
@Andrzej: deleting the libScreenMatchP
@Chang:
I was using method 2 from the two methods Chang mentioned above. And let's sort out what happened
1) To generate the tmplib, I followed your instruction using the sikuli-script.jar extracted libScreenMatchP
2) Changed all the path for libScreenMatchP
eng14:Sikuli-
eng14:tmplib amitp$ otool -L libScreenMatchP
libScreenMatchP
../../
@loader_
@loader_
@loader_
/opt/local/
@loader_
/System/
@loader_
/usr/lib/
/usr/lib/
3) I checked and I do have all those libraries inside my ../Frameworks folder
4) execute script that is at ../tmplib folder like this
java -d32 -Dpython.path=Lib/ -jar sikuli-script.jar test.sikuli
Error message:
java.lang.
Revision history for this message
|
#23 |
Also method 1 won't work neither for example:
open -a /Applications/
Will give you run time error from Sikuli IDE, it was complaining an imported library from POI.sikuli test suite was using lines like "import sikuli.Sikuli.*"
Also I don't think this method is scalable, since it is hard to catch the popup error message when you are running things in a batch.
Note that I had hundreds of testcases inside tens of testsuites running fine with the method 2 before upgrading to Sikuli 1.0, however I cannot get the tmplib working now.
I wonder if anyone got Sikuli 1.0 working in a command line mode by calling in the way that was working for older version of Sikuli:
java -d32 -Dpython.path=Lib/ -jar sikuli-script.jar test.sikuli
Note that the Lib was extracted from sikuli-script.jar and I was told the tmplib should have some lib and path needs to be fixed somehow to get it working. In the older version, the tmplib was automatically generated.
I got this method working for older version of Sikuli from this post:
Revision history for this message
|
#24 |
Try this package: http://
It should have all correct dylibs and jnilibs you need.
Revision history for this message
|
#25 |
I got it working libo. I will write some guide on it soon, meanwhile
if you want I can send u packed example for Python and Java.
Andrzej
Wysłane z iPhone'a
Dnia 2010-05-14 o godz. 21:16 libo
<email address hidden> napisał(a):
> Question #108878 on Sikuli changed:
> https:/
>
> libo posted a new comment:
> Also method 1 won't work neither for example:
>
> open -a /Applications/
> /Users/
>
> Will give you run time error from Sikuli IDE, it was complaining an
> imported library from POI.sikuli test suite was using lines like
> "import
> sikuli.Sikuli.*"
>
> Also I don't think this method is scalable, since it is hard to catch
> the popup error message when you are running things in a batch.
>
> Note that I had hundreds of testcases inside tens of testsuites
> running
> fine with the method 2 before upgrading to Sikuli 1.0, however I
> cannot
> get the tmplib working now.
>
> I wonder if anyone got Sikuli 1.0 working in a command line mode by
> calling in the way that was working for older version of Sikuli:
>
> java -d32 -Dpython.path=Lib/ -jar sikuli-script.jar test.sikuli
>
> Note that the Lib was extracted from sikuli-script.jar and I was told
> the tmplib should have some lib and path needs to be fixed somehow to
> get it working. In the older version, the tmplib was automatically
> generated.
>
> I got this method working for older version of Sikuli from this post:
>
> https:/
>
> --
> You received this question notification because you are a direct
> subscriber of the question.
Revision history for this message
|
#26 |
Wow, magical, all those libraries loaded now, it is only complaining that import sikuli.Sikuli.* is not valid, for older versions it works fine as "from python.
eng14:Sikuli-
ScreenMatchProxy loaded.
VDictProxy loaded.
ScreenMatchProxy loaded.
Exception in thread "main" Traceback (most recent call last):
File "/Users/
import search
SyntaxError: ("mismatched input '*' expecting set null", ('/Users/
Revision history for this message
|
#27 |
Thanks Andrzej, Chang's files also helps loading all the libraries. I look forward to your detailed guide, since I have no idea where Chang got those files under tmplib, since in the original post it only mentioned one library, but there are three in his tmplib.
Now it goes back to the question of the original post
The older version to import sikuli works fine
from python.
but not the new one
import sikuli.Sikuli.* is not valid to me.
In my test suite, I was calling another python library that implement all the routines, in which it has to import sikuli somehow.
I would also write a post on how to implement large scale testing framework using latest Sikuli 1.0 after I resolve this issue.
Revision history for this message
|
#28 |
I think from edu.mit.
Revision history for this message
|
#29 |
@libo:
The three jnilibs are copied from Sikuli-
Can you see if you have sikuli/ under Lib/? If not, you may extract the Lib from a wrong sikuli-script.jar.
Revision history for this message
|
#30 |
@Andrzej,
I changed the import to from edu.mit.
@Chang,
There is a sikuli under Lib, and I got that Lib extracted from the sikuli-script.jar under
/Applications/
Hopefully that is the one I am suppose to use.
Revision history for this message
|
#31 |
@Andrzej and @Chang,
I finally also got the import working from other python script, you need to use
from sikuli.Sikuli import *
instead of
from edu.mit.
or
import sikuli.Sikuli.*
Now start the hard work to convert all my old find() to new way of using them -- define object of Region or Screen first and call it within the object.
Thanks guys,
I promise to write a post on this migration experience and how to build large scaled testing framework using Sikuli
Libo
Revision history for this message
|
#32 |
@libo,
Congrats on figuring out the problem. :)
Looking forward to your post.
Revision history for this message
|
#33 |
One more thing though.
In sikuli 1.0 documentation, it mentioned that people can call find(), click() etc directly and by default sikuli will assume it is calling the one in Screen class. This works inside the IDE
However when you try to use it from a python script by importing
import sikuli.Sikuli.*
It only recognize those methods inside sikuli/Sikuli.py as global methods. When you try to use find() directly, it won't allow you. Why is that? Shouldn't both method works based on documentation?
Maybe I am missing something here. note that edu.mit.csail.uid is not packed inside Lib any more, they are now only java classes outside the Lib folder, so nothing else for me to include.
Revision history for this message
|
#34 |
edu.mit.csail.uid is a Java package, not for Jython. If you are using Jython scripts, you should use "from sikuli.Sikuli import *".
The global functions are constructed automatically when you import sikuli.Sikuli. It should work even without the IDE. (See the function initSikuli() in sikuli/Sikuli.py if you are interested in the details.)
Revision history for this message
|
#35 |
@Chang,
I read the source, and every python script that include the line of
from sikuli.Sikuli import *
should be able to call find() and click() etc.
I AM able to do that if importing directly from a sikuli script. However this is what not working:
If you write a sikuli script, and inside your test.py file, you import another python Class/module, and inside that class you try to use click() or find(), you won't be able to. Even if you imported sikuli inside that class too.
Revision history for this message
|
#36 |
Here is one example:
1) Make a file test.py inside test.sikuli
import sys
import unittest
import recorder
from sikuli.Sikuli import *
class Test(unittest.
def setUp(self):
def tearDown(self):
def test_direct_
a = find("PATH_
if a:
def test_indirect_
suite = unittest.
unittest.
2) Make a class file named recorder.py
from sikuli.Sikuli import *
class Recorder:
def __init__(self):
pass
def find_and_
a = find("PATH_
if a:
3) make sure you replace the PATH_TO_AN_IMAGE to any image locally
4) At the folder with all the setup folder (the files you sent to me) tmplib, Lib, sikuli-script.jar, Frameworks etc, run the following command:
java -d32 -Dpython.path=Lib/ -jar sikuli-script.jar test.sikuli
And as a result, you should see your first test passed, but the second test will fail with exception telling you "find" name error.
Revision history for this message
|
#37 |
I see.
It's a name space problem.
The initSikuli() function only exposes the functions to __main__. So, if you import sikuli from a module other than __main__, you won't be able to see the Region's methods as global functions.
It's a little tricky here. I need some time to think about an elegant way to fix this for modules.
Revision history for this message
|
#38 |
Thanks Chang, hopefully we can figure out a way to resolve this.
When I was using older version Sikuli, it worked out fine though, I haven't check the source on that version of Sikuli yet.
To build a large scale system, usually you won't write your methods inside testsuites, instead you write them inside a separate library (then many test cases and testsuites can use the same library methods) so this problem must be resolved to make Sikuli more suitable for larger systems.
In the mean while if we cannot resolve this, I will roll back to older version of Sikuli, which supports this feature
Revision history for this message
|
#39 |
@libo,
I have figured out a way to resolve this. :)
Here is the patch, http://
Revision history for this message
|
#40 |
How do you apply the patch?
------Original Message------
From: Tsung-Hsiang Chang
Sender: <email address hidden>
To: <email address hidden>
ReplyTo: <email address hidden>
Subject: Re: [Question #108878]: Write sikuli code in pure java?
Sent: May 15, 2010 9:05 AM
Question #108878 on Sikuli changed:
https:/
Tsung-Hsiang Chang posted a new comment:
@libo,
I have figured out a way to resolve this. :)
Here is the patch, http://
--
You received this question notification because you are a direct
subscriber of the question.
Sent via BlackBerry by AT&T
Revision history for this message
|
#41 |
You know since I am running command line mode on mac without IDE and we did a bunch of file copy and path changing stuff. I have Lib, Frameworks and tmplib under testing dir
------Original Message------
From: libo
Sender: <email address hidden>
To: <email address hidden>
ReplyTo: <email address hidden>
Subject: Re: [Question #108878]: Write sikuli code in pure java?
Sent: May 15, 2010 11:03 AM
Question #108878 on Sikuli changed:
https:/
libo proposed the following answer:
How do you apply the patch?
------Original Message------
From: Tsung-Hsiang Chang
Sender: <email address hidden>
To: <email address hidden>
ReplyTo: <email address hidden>
Subject: Re: [Question #108878]: Write sikuli code in pure java?
Sent: May 15, 2010 9:05 AM
Question #108878 on Sikuli changed:
https:/
Tsung-Hsiang Chang posted a new comment:
@libo,
I have figured out a way to resolve this. :)
Here is the patch, http://
--
You received this question notification because you are a direct
subscriber of the question.
Sent via BlackBerry by AT&T
You received this question notification because you are a direct
subscriber of the question.
Sent via BlackBerry by AT&T
Revision history for this message
|
#42 |
Hi Chang,
I know you and Raiman do not lack of oppertunities at all, but I have to ask. If you guys do ever need summer internship or a full time job, please let me know. I work for Telenav, which just went IPO last thursday(TNAV) and it is a fast growing company, we have server side, client side, and auto embeded applications, there are tons of oppertunities here, just let me know if you guys would be interested, it would be exciting to work with you guys
Libo
Sent via BlackBerry by AT&T
Revision history for this message
|
#43 |
My appologies, the above message is meant to send only to Chang, I was using my blackberry to reply to you name only but it ended up here, please delete it if possible
------Original Message------
From: libo
Sender: <email address hidden>
To: <email address hidden>
ReplyTo: <email address hidden>
Subject: [Question #108878]: Do you guys need any summer internship or a fulltime job ?
Sent: May 15, 2010 11:21 AM
Question #108878 on Sikuli changed:
https:/
libo proposed the following answer:
Hi Chang,
I know you and Raiman do not lack of oppertunities at all, but I have to ask. If you guys do ever need summer internship or a full time job, please let me know. I work for Telenav, which just went IPO last thursday(TNAV) and it is a fast growing company, we have server side, client side, and auto embeded applications, there are tons of oppertunities here, just let me know if you guys would be interested, it would be exciting to work with you guys
Libo
Sent via BlackBerry by AT&T
--
You received this question notification because you are a direct
subscriber of the question.
Sent via BlackBerry by AT&T
Revision history for this message
|
#44 |
@libo,
It seems there is no way to delete messages here...
Thanks for your kindness. I already have an internship this summer.
But maybe we will have opportunities to collaborate together. :)
To answer your question of how to apply the patch, just run "patch < sikuli-mod.patch" under Lib/sikuli.
Revision history for this message
|
#45 |
Awesome! the patch works, now it can recognize methods by import sikuli.Sikuli.* from imported modules.
Only note is that when the patch is asking you which file to patch, if you are running under Lib dir, you need to give your full path to current dir then add /sikuli/Screen.py and /sikuli/Sikuli.py
Thanks to Chang and Andrzej for helping me for this migration, I will make a new post on this thread to share the procedures on running Sikuli 1.0 under command line mode for large scale testing system.
Revision history for this message
|
#46 |
@libo,
Great! Looking forward to your post. :)
Revision history for this message
|
#47 |
Posted
"Building Large-scale Testing framework Using Sikuli 1.0 - Mac example "
https:/
Thanks to Chang and Andrzej for your help with this upgrade.
Can you help with this problem?
Provide an answer of your own, or ask Stuart Robinson for more information if necessary.