New project about sikuli(sikuli remote control)

Thanks for sharing this solution. Enix Shen, I noticed you also made the Sikuli XML-RPC server to run Sikuli IDE scripts remotely, also on GitHub.

I think next major step in this project you posted is how to make it work in Grid environment (maybe just a Sikuli tool only Grid) but the big idea is to get it to work in Grid mode integrated with Selenium / WebDriver. That is from a single Grid, you can run test that performs Selenium and Sikuli automation on same given node(s).

FYI, your approach here is similar to mine but implemented and used differently. Mine also maps basic functionality only:

http://code.google.com/p/simplesikuli

Yes, that would be a great step if we can integrate them with grid.

Thanks,
Enix

On Mon, Jun 11, 2012 at 3:11 AM, daluu <<email address hidden>
> wrote:

> Your question #200005 on Sikuli changed:
> https://answers.launchpad.net/sikuli/+question/200005
>
> daluu posted a new comment:
> Thanks for sharing this solution. Enix Shen, I noticed you also made the
> Sikuli XML-RPC server to run Sikuli IDE scripts remotely, also on
> GitHub.
>
> I think next major step in this project you posted is how to make it
> work in Grid environment (maybe just a Sikuli tool only Grid) but the
> big idea is to get it to work in Grid mode integrated with Selenium /
> WebDriver. That is from a single Grid, you can run test that performs
> Selenium and Sikuli automation on same given node(s).
>
> FYI, your approach here is similar to mine but implemented and used
> differently. Mine also maps basic functionality only:
>
> http://code.google.com/p/simplesikuli
>
> --
> You received this question notification because you asked the question.
>

I am using RobotFramework (http://code.google.com/p/robotframework/) remote libraries to accomplish the same thing.

Where I server Selenium/Sikuli keywords(methods) as remote RobotFramework Libraries (XML-RPC).

http://code.google.com/p/robotframework/wiki/RemoteLibrary

This question was expired because it remained in the 'Open' state without activity for the last 15 days.

To keep it visible

Hi,
Previously was using the Sikuli X.
After reading about this project, tried to use it.
I started the server on a mac as : java -jar server-mac-0.0.1-SNAPSHOT-j-with-dependencies.jar

Then when I tried to get the RemoteClient I get below error on the server console:
2012-07-26 16:50:55,737 [1350691283@qtp-566329178-0] INFO org.sikuli.remote.worker.SikuliWorker - Executing Sikuli Script Begins
java.io.IOException: Unable to find library META-INF/lib/libVisionProxy.dylib on classpath
 at com.wapmx.nativeutils.jniloader.DefaultJniExtractor.extractResource(DefaultJniExtractor.java:127)
 at com.wapmx.nativeutils.jniloader.DefaultJniExtractor.extractJni(DefaultJniExtractor.java:110)
 at com.wapmx.nativeutils.jniloader.NativeLoader.loadLibrary(NativeLoader.java:43)
 at org.sikuli.script.Finder.<clinit>(Finder.java:33)
 at java.lang.Class.forName0(Native Method)
 at java.lang.Class.forName(Class.java:264)
 at org.python.core.Py.loadAndInitClass(Py.java:895)
 at org.python.core.Py.findClassInternal(Py.java:830)
 at org.python.core.Py.findClassEx(Py.java:881)
 at org.python.core.packagecache.SysPackageManager.findClass(SysPackageManager.java:133)
 at org.python.core.packagecache.PackageManager.findClass(PackageManager.java:28)
 at org.python.core.packagecache.SysPackageManager.findClass(SysPackageManager.java:122)
 at org.python.core.PyJavaPackage.__findattr_ex__(PyJavaPackage.java:137)
 at org.python.core.PyObject.__findattr__(PyObject.java:863)
 at org.python.core.imp.import_name(imp.java:849)
 at org.python.core.imp.importName(imp.java:884)
 at org.python.core.ImportFunction.__call__(__builtin__.java:1220)
 at org.python.core.PyObject.__call__(PyObject.java:357)
 at org.python.core.__builtin__.__import__(__builtin__.java:1173)
 at org.python.core.imp.importFromAs(imp.java:978)
 at org.python.core.imp.importFrom(imp.java:954)
 at sikuli.Sikuli$py.f$0(/Users/akadimisetty/Documents/workspace/sikuli/jars/server-mac-0.0.1-SNAPSHOT-jar-with-dependencies.jar/Lib/sikuli/Sikuli.py:211)
 at sikuli.Sikuli$py.call_function(/Users/akadimisetty/Documents/workspace/sikuli/jars/server-mac-0.0.1-SNAPSHOT-jar-with-dependencies.jar/Lib/sikuli/Sikuli.py)
 at org.python.core.PyTableCode.call(PyTableCode.java:165)
 at org.python.core.PyCode.call(PyCode.java:18)
 at org.python.core.imp.createFromCode(imp.java:386)
 at org.python.core.util.importer.importer_load_module(importer.java:109)
 at org.python.modules.zipimport.zipimporter.zipimporter_load_module(zipimporter.java:161)
 at org.python.modules.zipimport.zipimporter$zipimporter_load_module_exposer.__call__(Unknown Source)
 at org.python.core.PyBuiltinMethodNarrow.__call__(PyBuiltinMethodNarrow.java:47)
 at org.python.core.imp.loadFromLoader(imp.java:513)
 at org.python.core.imp.find_module(imp.java:467)
 at org.python.core.PyModule.impAttr(PyModule.java:100)
 at org.python.core.imp.import_next(imp.java:715)
 at org.python.core.imp.import_name(imp.java:824)
 at org.python.core.imp.importName(imp.java:884)
 at org.python.core.ImportFunction.__call__(__builtin__.java:1220)
 at org.python.core.PyObject.__call__(PyObject.java:357)
 at org.python.core.__builtin__.__import__(__builtin__.java:1173)
 at org.python.core.imp.importAll(imp.java:998)
 at sikuli$py.f$0(/Users/akadimisetty/Documents/workspace/sikuli/jars/server-mac-0.0.1-SNAPSHOT-jar-with-dependencies.jar/Lib/sikuli/__init__.py:3)
 at sikuli$py.call_function(/Users/akadimisetty/Documents/workspace/sikuli/jars/server-mac-0.0.1-SNAPSHOT-jar-with-dependencies.jar/Lib/sikuli/__init__.py)
 at org.python.core.PyTableCode.call(PyTableCode.java:165)
 at org.python.core.PyCode.call(PyCode.java:18)
 at org.python.core.imp.createFromCode(imp.java:386)
 at org.python.core.util.importer.importer_load_module(importer.java:109)
 at org.python.modules.zipimport.zipimporter.zipimporter_load_module(zipimporter.java:161)
 at org.python.modules.zipimport.zipimporter$zipimporter_load_module_exposer.__call__(Unknown Source)
 at org.python.core.PyBuiltinMethodNarrow.__call__(PyBuiltinMethodNarrow.java:47)
 at org.python.core.imp.loadFromLoader(imp.java:513)
 at org.python.core.imp.find_module(imp.java:467)
 at org.python.core.imp.import_next(imp.java:713)
 at org.python.core.imp.import_name(imp.java:824)
 at org.python.core.imp.importName(imp.java:884)
 at org.python.core.ImportFunction.__call__(__builtin__.java:1220)
 at org.python.core.PyObject.__call__(PyObject.java:357)
 at org.python.core.__builtin__.__import__(__builtin__.java:1173)
 at org.python.core.imp.importFromAs(imp.java:978)
 at org.python.core.imp.importFrom(imp.java:954)
 at org.python.pycode._pyx0.f$0(<string>:1)
 at org.python.pycode._pyx0.call_function(<string>)
 at org.python.core.PyTableCode.call(PyTableCode.java:165)
 at org.python.core.PyCode.call(PyCode.java:18)
 at org.python.core.Py.runCode(Py.java:1261)
 at org.python.core.Py.exec(Py.java:1305)
 at org.python.util.PythonInterpreter.exec(PythonInterpreter.java:206)
 at org.sikuli.remote.worker.SikuliWorker.work(SikuliWorker.java:47)
 at org.sikuli.remote.server.servlet.ScriptServlet.doPost(ScriptServlet.java:28)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
 at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
 at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
 at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
 at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
 at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
 at org.mortbay.jetty.Server.handle(Server.java:326)
 at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
 at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945)
 at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
 at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
 at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
 at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
 at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
[info] Mac OS X utilities loaded.

I also had opencv installed separately.
And I was running the test from eclipse.

Any help would be very highly appreciated.

Thanks in Advance,
Anil

ok..I found libVisionProxy.dylib in my Application folder .
So i extracted the jar file and copied the .dylib to the META-INF/lib folder and jar'd it again and the above exception is gone.

Now, on running the client program i see the below exception though:-

Exception in thread "main" org.sikuli.remote.script.ScriptExcuteFailedException: java.lang.UnsatisfiedLinkError: /Applications/Sikuli-IDE.app/Contents/Frameworks/libVisionProxy.dylib: dlopen(/Applications/Sikuli-IDE.app/Contents/Frameworks/libVisionProxy.dylib, 1): Library not loaded: /usr/local/lib/libopencv_core.2.2.dylib
  Referenced from: /Applications/Sikuli-IDE.app/Contents/Frameworks/libVisionProxy.dylib
  Reason: image not found
 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
 at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
 at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
 at org.sikuli.remote.script.SikuliScript.retriveError(SikuliScript.java:68)
 at org.sikuli.remote.client.Client.call(Client.java:35)
 at org.sikuli.remote.client.SikuliAction.doAction(SikuliAction.java:73)
 at org.sikuli.remote.client.RemoteScreen.setMinSimilarity(RemoteScreen.java:65)
 at org.sikuli.remote.client.RemoteScreen.setDefaultMinSimilarity(RemoteScreen.java:69)
 at org.sikuli.remote.client.RemoteScreen.<init>(RemoteScreen.java:18)
 at Sikuli1.main(Sikuli1.java:14)

Any insight will be highly appreciated.

You need a valid OpenCV 2.2installation on your system

Looks like I have opencv2.4 in my machine

Is there a way to tell the sikuli-remote-control to pick that version?
if i am compiling and building it locally?

Just wanted to mention that sikuli remote control has API close to WebDriver. I haven't looked at the codebase myself yet. But makes me wonder, if it could be reworked to actually conform to the WebDriver JSONWireProtocol so that the server that interfaces to Sikuli can be consumed by any WebDriver client language binding (not requiring use of sikuli remote control Java client nor the Ruby client https://github.com/enix12enix/sikulirc).

Some closer to this that I attempted:

https://answers.launchpad.net/sikuli/+question/234883
https://github.com/daluu/SikuliDriverServer