[1.0] integrating selenium with sikuli ide

Asked by ABET on 2013-04-09

Is there a way to integrate the selenium (from jar file) with sikuli ide?
was trying this:
import selenium
from selenium import webdriver

and get a bunch of errors.

Tried using the python version of selenium, and getting a bunch of errors too
import selenium
from selenium import webdriver
But if I try this on python directly, it seemed to work.

Question information

English Edit question
Sikuli Edit question
RaiMan Edit question
Solved by:
Last query:
Last reply:

There was a similar question on the forum recently:

integrate WebDriver (Selenium 2.0) into Sikuli IDE

I think that answer may have information you requested.

ABET (abetv) said : #2

Yeah. I saw those, didn't help much.

RaiMan (raimund-hocke) said : #3

If you want to use the features from selenium.jar (that is the Java API), you have to put the jar in the Java class path in all cases.

If you want to use the Java API from within a Sikuli script (Jython level), you have to import the Selenium classes this way:

import selenium.xxx.yyy.webdriver as WD

now WD is a class shortcut

and then use the webdriver methods as you do on the Java level:

wd = WD() # a new webdriver object

ABET (abetv) said : #4

Y. I tried that as well but got errors.

this is what I did:
- copied the selenium.jar file over to the IDE dir /Applications/Sikuli-IDE.app/Contents/Resources/java/
- started sikuli with a classpath exported:
> export HOME=/Applications/Sikuli-IDE.app/Contents/Resources/java
> export CLASSPATH=/Applications/Sikuli-IDE.app/Contents/Resources/java
> java -cp $CLASSPATH -jar $HOME/sikuli-script.jar -i
>>> import selenium
 (got err: no module named selenium)
>>> import org.openqa.selenium as WD
 (also got an err: no module named openqa)

ABET (abetv) said : #5

was able to to get the java cm to load the selenium jar file. did not know that the -cp is ignored if -jar is passed.

java -cp $SIKULI_HOME/selenium.jar:$SIKULI_HOME/sikuli-script.jar org.python.util.jython -i
> from sikuli import *
> from org import *
> from org.openqa import *
> import org.openqa.selenium.firefox.FirefoxDriver as FF

Then I can launch ff using the selenium call
> f = ff()

problem I have now is,
 how can I make sikuli know about that browser that just launched using the driver so I can get both selenium and sikuli to play well?
I tried using swtichApp and App.focus, but no luck....
(OR would this be a different ticket?? ) :)

RaiMan (raimund-hocke) said : #6

java -cp $SIKULI_HOME/selenium.jar $SIKULI_HOME/sikuli-script.jar -i

should work.

at the interactive prompt:
> from sikuli import * # not needed - already done
> from org import * # does not make any sense
> from org.openqa import * # does not make any sense
> import org.openqa.selenium.firefox.FirefoxDriver as FF # this is the winner ;-)
>ff = FF() # should give you the browser object

App("text from the FF window title").focus() # should bring the FF window to front
ffWin = Region(App.getFocusedWindow()) # should be the region of the FF window

now you should have the region for Sikuli (ffWin) and the browser object for Selenium (ff)

ABET (abetv) said : #7

Hi Raiman, that does not seem to work. I get an error.

java -cp $SIKULI_HOME/selenium.jar $SIKULI_HOME/sikuli-script.jar -i
Exception in thread "main" java.lang.NoClassDefFoundError: /Applications/Sikuli-IDE/app/Contents/Resources/java/sikuli-script/jar
Caused by: java.lang.ClassNotFoundException: .Applications.Sikuli-IDE.app.Contents.Resources.java.sikuli-script.jar
 at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
 at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:247)

RaiMan (raimund-hocke) said : #8

sorry for the mistake - must be

java -cp $SIKULI_HOME/selenium.jar -jar $SIKULI_HOME/sikuli-script.jar -i

ABET (abetv) said : #9

Hi Raiman, that was the command that I tried at the beginning and it does not work either. The sikuli would start, but I would not be able to use the selenium piece. Java ignores the class path when the jar option is passed.

If u could please, write the steps down after trying it. What I am trying to do here, is to run it in interactive mode first to see how it works before implementing as a script. So I can start sikuli with just providing a selenium jar files and be able to execute selenium Apis. Launch browser using selenium we driver and use sikuli to focus on the region. I would then use selenium to trigger the events and use sikuli to do the results checking ( or probably other methods that seems easier for sikuli). If u have a sample like his in python I would love to see it. Thnx much

Best RaiMan (raimund-hocke) said : #10

Uuups, yes you are right, but this is not Java problem, it is some Sikuli problem, that blocks other org.xxx from the class path (same problem is with SQL-database drivers).

so this works for me:
export CLP=/Users/rhocke/Downloads/selenium-2.32.0/selenium-java-2.32.0.jar:/Applications/Sikuli-IDE.app/Contents/Resources/Java/sikuli-script.jar
java -cp $CLP org.sikuli.script.SikuliScript -i
>import org.openqa.selenium.firefox.FirefoxDriver as FF #works

the followup problem is, that the needed dependent java stuff for selenium is not found (com/google/common/base/Function in this case)

I downloaded google-collections-0.8.jar and put it in classpath too, which makes the above import successful.

But then
ff = FF()

yields an error, that looks like google-collections-0.8.jar is the wrong jar version.

at this point I stop now (using Selenium is not my focus): the above approach principally works, but now you have to find out, which selenium jar is really needed and what dependent stuff needs to be in classpath too.

ABET (abetv) said : #11

Thanks RaiMan, that solved my question.

ABET (abetv) said : #12

thanks RaiMan. I know selenium is not your focus on this forum, but you did help me move forward to understand how sikuli can work with selenium with those steps.

Looking forward to the next sikuli release with better OCR and text recognition is released soon.
Can you please update the forums on when you think it will be released or at least update r930 with the latest OCR and text recognition support?

RiSol (rinasolomon) said : #13

I succeeded to run Selenium and Sikuli code using Sikuli IDE 1.0.0 based on this detailed conversation and what is my configuration.

in sikuli-ide.cmd:

set SELENIUM_HOME=C:\selenium-2.33.0\
echo +++ trying to start Sikuli IDE with %JAVA%
"%JAVA%\java.exe" %PARMS% -cp "%SELENIUM_HOME%libs\*;%SELENIUM_HOME%selenium-java-2.33.0.jar;%SIKULIX_HOME%sikuli-script.jar;%SIKULIX_HOME%sikuli-ide.jar" org.sikuli.ide.SikuliIDE %SPARMS% %SIKULI_USERPARMS%

in sikuli ide:

import org.openqa.selenium.firefox.FirefoxDriver as FF
import java.util.concurrent.TimeUnit as TimeUnit
import org.openqa.selenium.By as By

driver = FF()
baseUrl = "http://XXXX:8080/"
driver.get(baseUrl + "/XXX-app/")
click("Save File.png")

RaiMan (raimund-hocke) said : #14

Thanks for documenting

Daniel Kurtz (dkurtz-p) said : #15

OK, I recognize that this thread is old old old old seriously old, but I'm not finding anything more recent or more comprehensive. I've updated the .cmd file as specified (changing all references to Se2.33.0 to Se2.45.0) but when I try to run the sample script I get:

     [error] ImportError ( No module named openqa )

Any guidance appreciated.

Daniel Kurtz (dkurtz-p) said : #16

OK, scratch last comment. CMD file was not executing the modified command. It does now, and fails.

CMD file:
echo +++ running this Java
java -version
set SELENIUM_HOME=C:\selenium-2.45.0\
echo +++ trying to run SikuliX
echo +++ using: %PARMS% -cp "%SELENIUM_HOME%libs\*;%SELENIUM_HOME%selenium-java-2.45.0.jar;%SIKULIX_HOME%sikuli-script.jar;%SIKULIX_HOME%sikuli-ide.jar" org.sikuli.ide.SikuliIDE %SPARMS% %SIKULI_USERPARMS%
java %PARMS% -cp "%SELENIUM_HOME%libs\*;%SELENIUM_HOME%selenium-java-2.45.0.jar;%SIKULIX_HOME%sikuli-script.jar;%SIKULIX_HOME%sikuli-ide.jar" org.sikuli.ide.SikuliIDE %SPARMS% %SIKULI_USERPARMS%

Error message:

+++ JAVA_HOME specified but not useable "C:\Program Files\Java\jre1.8.0_45"
+++ looking for Java in system path and then in standard places
+++ running this Java
java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) Client VM (build 25.45-b02, mixed mode, sharing)
+++ trying to run SikuliX
+++ using: -Xms64M -Xmx512M -Dfile.encoding=UTF-8 -Dsikuli.FromCommandLine -cp "C:\selenium-2.45.0\libs\*;C:\selenium-2.45.0\selenium-java-2.45.0.jar;C:\Sikuli1
.1\sikuli-script.jar;C:\Sikuli1.1\sikuli-ide.jar" org.sikuli.ide.SikuliIDE
Error: Could not find or load main class org.sikuli.ide.SikuliIDE

Again, any guidance appreciated.


Maria Haris (mariaharis) said : #17

I have tried to configure the same settings as mentioned by Risol but I am not able to run my cmd file. It just opens and closes without showing any errors.I am trying to use selenium 3 in Sikuli 1.1.0. Are there some extra configurations which need to be made to make it work ?

Maria Haris (mariaharis) said : #18

I have tried to configure the same settings as mentioned by Risol but I am not able to run my cmd file. It just opens and closes without showing any errors.I am trying to use selenium 3 in Sikuli 1.1.0. Are there some extra configurations which need to be made to make it work ?