SikuliX

Using Sikuli for layout/visual testing

Asked by Kandada Boggu

We have a integration test suite built on top of Cucumber. Even though we were able to automate the UI behavior testing we haven't been able to automate the UI appearance testing.

I am wondering if I can use Sikuli for this purpose. Ideally this is the approach I want to take

Test Data preparation
==============
- Capture the screen shots for each screen and store the images in a known location. This is a one time process.

Visual testing
========
1) Launch the screen for visual test.

2) Compare the screen to the expected screen.

I haven't figured how to perform the step 2. How do I compare the entire screen to a stock image?

This is one of the approaches I am thinking. Let me know if I am on the right track.

import sys
image = sys.argv[1]
switchApp("Firefox.app")
if exists(Pattern(image).similar(0.91)):
 print "success"
else:
 print "failure"

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
Kandada Boggu
Solved:
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said : 		#1

The approach is ok.

This snippet has to be run from command line, since only in this case you can use sys.argv.

Be aware, that each Sikuli test you are running this way takes about 5 seconds startup time. If this is a problem, you have to set up a Sikuli environment, that is up and waiting and runs the tests on request. A simple solution can be setup with XML-RPC (come back if interested).

It is recommended to not capture the whole screen in your step "Test Data preparation". Only capture the screen area your app is responsible for (in your case the content of a Firefox window). This might be done with support of a Sikuli script together with the respective Python modules for file handling. In the test (step 2) on the other hand, you should restrict the search to the Firefox window:

# only the relevant lines
ff = App("Firefox")
ff.focus() # switches to FF
content = ff.window() # the FF window
if content.exists(Pattern(image).similar(0.91)):

This makes your tests robust and faster.

BTW:
Since you seem to test a web app: do you use webdriver for the behavior testing?

Revision history for this message
Kandada Boggu (kandadaboggu) said : 		#2

Thanks for the reply. I was thinking about capturing just the FF content and not the entire desktop. I will surely use the suggestions about eager loading.

We use Cucumber + Watir (with Webdriver) for BDD.

Revision history for this message
RaiMan (raimund-hocke) said : 		#3

It would be nice, if you give some comments on your selected solution of "eager loading" and how you integrated this into your Cocumber scenarios, when it is ready.