Windows on Amazon EC2: Selenium and Sikuli conflict??

Asked by Min-cheng HUANG

Hello,

Currently, I am running my testing by combining Selenium and Sikuli. But, recently, I am facing the some problems that seems to be conflict between Selenium and Sikuli.

Usually, I use Selenium to open browser, direct to other pages, and close browser. And, I use Sikuli to do the action, such as click or verify Image.

The problem is that, the whole process seems to be stuck and waiting for Selenium. For example, I use Sikuli to finish verifying image, and Selenium is going to close browser. And, the whole browser just stuck there.

At first, I thought it would be Selenium Issue. But, an interesting thing happened. When I minimized the browser, the whole process suddenly continued to work. So, this makes me to think that, maybe, Sikuli is still controlling something or verifying something.

So, I am wondering whether Sikuli still holds some resources which will affect the work of Selenium.

Environment:
OS: windows server 2008
Browser: Firefox 8,
Jar: Selenium server 2.14, Sikuli 1.03

Also, I would like to ask whether Remote Desktop Connection will affect the work of Sikuli which is running on the remote machine.

Thanks a lot for any kindly help, and sorry for the unclear question.

PS: the same code seems to work fine in the WIN7 environment

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
Min-cheng HUANG
Solved:
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said :
#1

How do you use Sikuli together with Selenium? (Java coding or Sikuli scripts from command line or ???)

Could you give an example?

Revision history for this message
Min-cheng HUANG (mhuang-a) said :
#2

In Java.

The example could be:

1. FirefoxDriver driver = new FirefoxDriver(); //open a browser by Selenium
2. driver.get("http://www.google.com"); //go to google.com by Selenium
3. Screen s = new Screen();
4. s.exists("Some pattern to google search button", 30) // click search button by using Sikuli
5. driver.get("http://www.facebook.com"); // go to facebook.com by Selenium

The thing happened at the time when the Selenium switched to Sikuli as Sikuli switched to Selenium. So, it's Step 2 to Step4, and Step 4 to Step 5.

And, it's random. I found this issue when I ran my code under win server 2008. Java 7

The problem I am thinking about might be "OS", or "Race condition"

OS: because it seems the code runs pretty okay in other OS, win7 or win xp, however, it's JAVA code. It should not matter which environment the code runs.

Race condition: It seems that both APIs use same class. I am wondering whether Selenium code will invoke Sikuli to check screen. When my code freeze, I have no idea what it is waiting for. But, the solution to make the code continue to work is either I can minimized the browser or move other window to overlay the browser, then it works again.

So, it's like to change the current screen status which makes me to think about Sikuli is finding something on the current screen. And, to change the current screen status make Sikuli to give up searching. Because it makes no sense that Selenium will detect what is happening on the current screen, I would guess that it's Sikuli trying to do something at the same time.

I can take a video for the issue and that would be easier to understand the problem.

Thanks a lot for your kindly help.

Revision history for this message
Min-cheng HUANG (mhuang-a) said :
#3

By the way,

The API I am using:

selenium-server-standalone-2.15.0.jar
sikuli-script.jar of Sikuli x1.0-rc3

Revision history for this message
RaiMan (raimund-hocke) said :
#4

If you are really using Java 7:

This is currently not supported by Sikuli X-1.0rc3.

It needs JRE 6.

In a JRE 6 environment your code should work a expected:
in the above step 4 Sikuli will wait max 30 seconds for the image to come up and then return to your code, so step 5 is executed.

I am sure, that there are no interferences between Selenium and Sikuli as you mentioned above. They both do not know anything about each other, when used the way you do.

Revision history for this message
Abhinav (abhinav1708) said :
#5

I am successfully using Selenium and Sikuli together and it generates no issues

Maybe switching to java 6 may solve the problem

Revision history for this message
Min-cheng HUANG (mhuang-a) said :
#6

Hey Raiman,

I am thinking that, too. I will see what happens after switching back to Java 6.

Hey Abhinav,

Could you tell me what OS you are running both api?

Revision history for this message
Min-cheng HUANG (mhuang-a) said :
#7

The thing still happens even I used Jre 6.

Here is the youtube video you can check out.

http://www.youtube.com/watch?v=OrJERa41ybk
http://www.youtube.com/watch?v=CPA3tuNkE1s (start from 00:30)

I still cannot figure out why...

Revision history for this message
RaiMan (raimund-hocke) said :
#8

Put the

Screen s = new Screen();

before any Selenium code.

Revision history for this message
Min-cheng HUANG (mhuang-a) said :
#9

@RaiMan:
Actually, I instantiated Screen before Selenium code. I am thinking whether it would be environment issue, since it works fine on WIN7.

Revision history for this message
RaiMan (raimund-hocke) said :
#10

Ok, then what is the difference between your Win7 situation and that on Win Server?

Revision history for this message
Min-cheng HUANG (mhuang-a) said :
#11

It's Amazon EC2 instance, whose OS is windows server 2008.

The java version was Jre 7 before, but, as I mention above, I just downgraded the Java version to Jre 6. What else information I can provide to you?

Thanks a lot for your help.

Revision history for this message
RaiMan (raimund-hocke) said :
#12

Sorry, I do not have any experiences with EC2 based solutions. But as far as I know: here we are talking about some kind of virtualization, which changes the situation for Sikuli a lot (real screen / virtual screen).

Revision history for this message
Abhinav (abhinav1708) said :
#13

Visit this blog for more information about sikuli and selenium integration
http://selesik.blogspot.com/

Revision history for this message
Min-cheng HUANG (mhuang-a) said :
#14

@Raiman

That could be the reason, since I have real screen locally and use remote desktop connection to EC2 instance...

@Abhinav
Thanks a lot for the info. I will check it out :)

Revision history for this message
daluu (cuuld) said :
#15

On a related note, I don't recommend remote desktop connections to do remote automated testing. If you must login and check desktop, etc. use VNC. On EC2, that means you have to install VNC on the virtual machine, and on the EC2 config, open up VNC port 5900 (or other desired port) though you can restrict IP range that is able to access machine for additional security, and set up a VNC password as well.

VNC works better and you don't have to keep the remote desktop connection open (VNC can be closed and automation still runs).

And for debugging Windows Server 2008 issue, would be ideal to compare with a local installation of win 2008 vs EC2 to make sure is not EC2 problem. If you don't have license for win 2008, install a trial version for debugging.