[RC3] Sikuli IDE's unit test feature and the alternatives

Created by RaiMan on on 2011-11-09
Keywords:
Last updated by:
RaiMan on on 2013-10-27

***** Section relevant for RC3
based on questions like:
https://answers.launchpad.net/sikuli/+question/178068
https://answers.launchpad.net/sikuli/+question/177954
https://answers.launchpad.net/sikuli/+question/176005

and the known restrictions/bugs in the feature:

https://answers.launchpad.net/sikuli/+question/173217 / bug 867636

-- import not supported as expected
-- tests performed in reverse order
-- hangs when tests have syntax errors

*** So for more sophisticated scenarios use Python's unittest feature directly and run your scripts normally in IDE or with option -r from command line.

***** end section for RC3 only

***** the rest is valid too for version 1.0.1+ ----------------------------------------

For the basics on the following examples have a look at the docs at
http://docs.python.org/library/unittest.html

Take Care: things marked as available 2.6 or later might not be available in Sikuli's Jython 2.5.2

*** backport unittest2
There is a package unittest2 available in the net, that makes it possible to use features from 2.6 and 2.7 (e.g. skip tests) in Python prior versions. Since it is written in Python language only, it can be used with Sikuli.

**** a template for unit test cases separated in .sikuli scripts

(you might put everything in one script as well ;-)

--- in each of your modUnitTestx.sikuli

at the beginning of each script put:

from sikuli import *
import unittest

class UnitTestX(unittest.TestCase):

indent the rest of the script one level, so that all the def's belong to your TestCase class.

In this class put a bunch of "def testxxxx(self):" and may be setUp and tearDown.

so for this example you might have:
# script modUnitTestA.sikuli
class UnitTestA(unittest.TestCase):
    def setUp(self):
        popup("setting things up")
    def test1(self):
        popup("running test1")
    def test2(self):
        popup("running test2")
    def test3(self):
        popup("running test3")
    def tearDown(self):
        popup("tearing things down")

--- create a mainTest.sikuli

as a base template, to get a feeling:

import unittest

from modUnitTestA import * # supposing, scripts live in same directory

suite = unittest.TestLoader().loadTestsFromTestCase(UnitTestA) # see comment
unittest.TextTestRunner(verbosity=2).run(suite)

-- comment
loads all def()'s from class UnitTestA which is in modUnitTestA.sikuli.

**** a test runner, that produces HTML output
see: https://answers.launchpad.net/sikuli/+question/176005

I have a modified version, that stores a screenshot on failure, that you can look at with a link in the HTML result page.
download: https://dl.dropboxusercontent.com/u/42895525/HTMLTestRunner.zip

**** a tip for Hudson/Jenkins (from: https://launchpad.net/~j-the-k)

I recommend Jython/Python unittests instead of the sikuli-special version.

I use an Python TestRunner that outputs JUnit-XML testresultfiles that can e.g. be used by Hudson/Jenkins.

You can find a working xmltestrunner here:

http://pypi.python.org/pypi/XmlTestRunner/0.16654

You can use it like this:

from xmlrunner import *
import unittest

class MyTest(unittest.TestCase):
    def setUp(self):
        // setUp

    def testMyTest(self):
        // test

    def tearDown(self):
        // tearDown

suite = unittest.TestLoader().loadTestsFromTestCase(MyTest)
result = XMLTestRunner(file("unittest.xml", "w")).run(suite)

**** Sikuli and RobotFrameWork
http://blog.mykhailo.com/2011/02/how-to-sikuli-and-robot-framework.html

*** to be continued