How to include Sikuli in other python / jython programs

Asked by Komarue on 2010-04-29

I am not sure if this question has been answered before, but are you able to / how do you write a python script that can make use of the sikuli find / click / doubleClick functions in another python app? Just curious. Or perhaps via Jython or Java...

Ross

Question information

Language:
English Edit question
Status:
Expired
For:
Sikuli Edit question
Assignee:
No assignee Edit question
Last query:
2010-05-12
Last reply:
2010-05-28
RaiMan (raimund-hocke) said : #1

I got the following information from the developers (I myself did not try this until now ;-)

---------------
(1) The steps to use Sikuli.py without Sikuli IDE:
 1. add sikuli-script.jar into Java's CLASSPATH
 2. add -Dpython.home=sikuli-script.jar as a parameter of launching java
 3. add "from python.edu.mit.csail.uid.Sikuli import *" in your python code. (Note that the package name will be changed in 0.10.)

(2)
Sikuli need some opencv libraries in tmplib/.
Try to add tmplib/ to %PATH% before running Jython.

(3)
In theory, you can import the other xxx.sikuli/xxx.py and call its functions. A tricky step is that you need to call setBundlePath() before using the image files inside the other .sikuli bundle, otherwise Sikuli's functions doesn't know where to find the images.
We will try to make this easier in the future versions.
-----------------

Hope this helps a little down the road. Feedback about your experiences and findings are very much appreciated.

Komarue (ross-hendrickson) said : #2

Thanks for the speedy reply. I had seen your question/response while searching but wasn't sure if it was the best approach. Just to confirm, these steps are so you can write a java program that includes Jython + Sikuli so that it can run Sikuli functions?

I should have more time to work on this tomorrow. My supervisor is interested in using Sikuli to do JUnit testing. Hmm maybe a Sikuli IDE plugin to Eclipse would be in order some time here. That would be Snazzy. I am happy to see OpenCV being abstracted and made more usable for a specific task. Great work Sikuli devs!

RaiMan (raimund-hocke) said : #3

As far as I understood, you have to write something with python syntax, that is afterwards run by the jython environment (python on java), so you can use the Sikuli defined classes and methods in your program directly without using the Sikuli IDE.

Komarue (ross-hendrickson) said : #4

Alright,

I wrote a simple Java based test using code scraped from the Sikuli source code (ScriptRunner.java). The error I get when I run this below and I could use some help figuring out what I did wrong. I've tried it using Java as well as using jython.

"Exception in thread "main" Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named python"
---------------------------------------------------------

Steps taken so far.
Verified Sikuli can be run via command line

java -cp sikuli-script.jar org.python.util.jython
>>> from sikuli.Sikuli import *
ScreenMatchProxy loaded.
VDictProxy loaded.
ScreenMatchProxy loaded.

sikuli-script.jar is part of my build path
VM argument is in place
-Dpython.home=sikuli-script.jar

------------------Jython Attempt-------------------------
Jython project, sikuli-script.jar in build path, VM argument set

import os
from edu.mit.csail.uid.SikuliScript import *

wd = os.getcwd()
print wd
setBundlePath(wd)
s.click("1273690410344.png")
type("notepad" + Key.ENTER)
wait("1273690450266.png", 2000)
click("1273690450266.png")
type("Hello World")

I get the following error in console "C:\Users\XXXXXXXt\workspace\Sikuli\sikuliScripts.sikuli
Traceback (most recent call last):
  File "C:\Users\XXXXXXX\workspace\Sikuli\sikuliScripts.sikuli\sikuliScripts.py", line 6, in <module>
    setBundlePath(wd)
NameError: name 'setBundlePath' is not defined"

if I change the import to just
from edu.mit.csail.uid import *
I do get
Win32Util loaded.
ScreenMatchProxy loaded.
VDictProxy loaded.
ScreenMatchProxy loaded.
before the script dies on the setBundlePath() (if that is removed it dies on the click())

---------------------Java Class---------------------------

import java.io.File;
import java.io.IOException;
import java.util.Arrays;
import java.util.Iterator;
import java.util.LinkedList;

import org.python.util.PythonInterpreter;

public class MagicClass {

 private static LinkedList<String> _headers;

 public static void main(String[] args){

  String dotSikuliPath = "C:\\Users\\Light\\workspace\\Sikuli\\sikuliScripts.sikuli\\";
  String[] h = new String[]{
           "from python.edu.mit.csail.uid.Sikuli import *",
           "setThrowException(False)",
           "setShowActions(False)"
        };
  _headers = new LinkedList<String>(Arrays.asList(h));
  try{
   runPython(dotSikuliPath);
  }catch(IOException e){
   System.err.println(e.toString());
  }
 }

 public static void runPython(String dotSikuliPath) throws IOException{
       PythonInterpreter py = new PythonInterpreter();
       Iterator<String> it = _headers.iterator();
       while(it.hasNext())
          py.exec(it.next());
       py.exec("setBundlePath('" + dotSikuliPath + "')");
       File pyFile = getPyFrom(dotSikuliPath);
       py.execfile(pyFile.getAbsolutePath());
    }

 private static File getPyFrom(String dotSikuliPath) throws IOException{
       String name = new File(dotSikuliPath).getName();
       String prefix = name.substring(0, name.lastIndexOf('.'));
       return new File(dotSikuliPath + "/"+ prefix + ".py");
    }

}

Tsung-Hsiang Chang (vgod) said : #5

I notice that you are using the old code..
The new sikuli package name should be sikuli.Sikuli in Jython. The old python.edu.mit.csail.uid.Sikuli is obsoleted.

Komarue (ross-hendrickson) said : #6

Sorry I should have mentioned that I did try

from sikuli.Sikuli import * and got

Traceback (most recent call last):
  File "C:\Users\Light\workspace\Sikuli\sikuliScripts.sikuli\sikuliScripts.py", line 2, in <module>
    from sikuli.Sikuli import *
ImportError: No module named sikuli

Komarue (ross-hendrickson) said : #7

This is what eclipse is using to execute the jython script

"C:\Program Files (x86)\Java\jdk1.6.0_20\bin\javaw.exe" -classpath
"C:/jython2.5.1/jython.jar;C:\Users\Light\.eclipse\org.eclipse.platform_3.4.0_248562372\plugins\org.python.pydev_1.5.7.2010050621\PySrc\pydev_sitecustomize;C:\Users\Light\workspace\Sikuli\src;C:\Program Files (x86)\Sikuli\sikuli-script.jar;C:\jython2.5.1\Lib;C:\jython2.5.1\Lib\site-packages;__classpath__;__pyclasspath__;C:\Program
Files (x86)\Java\jre6\lib"
"-Dpython.path=C:\Users\Light\.eclipse\org.eclipse.platform_3.4.0_248562372\plugins\org.python.pydev_1.5.7.2010050621\PySrc\pydev_sitecustomize;C:\Users\Light\workspace\Sikuli\src;C:\ProgramFiles
(x86)\Sikuli\sikuli-script.jar;C:\jython2.5.1\Lib;C:\jython2.5.1\Lib\site-packages;__classpath__;__pyclasspath__;C:\Program
Files (x86)\Java\jre6\lib" -Dpython.home=sikuli-script.jar
org.python.util.jython
C:\Users\Light\workspace\Sikuli\sikuliScripts.sikuli\sikuliScripts.py

Tsung-Hsiang Chang (vgod) said : #8

If you can run "java -cp sikuli-script.jar org.python.util.jython" and import sikuli.Sikuli.* successfully, it should be fine to run your Java code. So, I think the problem is related to the paths you define. Try removing python.path and python.home, maybe you don't need them.

Komarue (ross-hendrickson) said : #9

That was the key I needed for the python approach I will continue working on getting the Java to work.

For everyone else who has this same question, if you are trying to run Sikuli inside of Eclipse using PyDev you need to set up a new Interpreter that points to sikuli-script.jar NOT your jython installation.

Libo Cao (libocao) said : #10

After the upgrade to 1.0, I start having this exception (with the modification of import sikuli.Sikuli.* ):

Exception in thread "main" Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "../../Sikuli-Mac-Lib_1.0/Lib/sikuli/Sikuli.py", line 165, in <module>
    initSikuli()
  File "../../Sikuli-Mac-Lib_1.0/Lib/sikuli/Sikuli.py", line 160, in initSikuli
    __main__.SCREEN = Screen()
  File "../../Sikuli-Mac-Lib_1.0/Lib/sikuli/Screen.py", line 15, in __init__
    r = JScreen().getBounds()
 at edu.mit.csail.uid.EventManager.createEventManager(Native Method)
 at edu.mit.csail.uid.EventManager.<init>(EventManager.java:37)
 at edu.mit.csail.uid.Region.init(Region.java:55)
 at edu.mit.csail.uid.Screen.initBounds(Screen.java:79)
 at edu.mit.csail.uid.Screen.<init>(Screen.java:93)
 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
 at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
 at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
 at org.python.core.PyReflectedConstructor.constructProxy(PyReflectedConstructor.java:163)

java.lang.UnsatisfiedLinkError: java.lang.UnsatisfiedLinkError: edu.mit.csail.uid.EventManager.createEventManager()J

Tsung-Hsiang Chang (vgod) said : #11

@libo: What platform are you running? I guess you don't remove old 0.9 files before install 0.10.

Libo Cao (libocao) said : #12

Chang,

I was running on Mac 10.6 platform.

I do not remove my old 0.9, just rename them so that I can roll back when needed, why is that a problem? Command line mode states which library u want to use

Launchpad Janitor (janitor) said : #13

This question was expired because it remained in the 'Open' state without activity for the last 15 days.