SikuliX

HTML reports shows pass even if the test Fails in Sikuli

Asked by Vishal singh

Hi
I am new to sikuli and currently automating my Desktop application
I am writing a script to check if the image is present on the application or not and generating the HTML report
But the report always shows the tests pass even if i close the application to check if it shows fail or not

Please tell me how to solve this error
Thank you in advance

Here is the code:
import os
import HTMLTestRunner
import unittest
import sys
import logUtils
logger = logUtils.createLog(r'C:\Tools\td5_gui_automation\Test_logs\toolbar_test.log')

class Util(unittest.TestCase):

    def setUp(self):
        # setup the test environment needed to execute the tests in this function
        try:
            TD5 = os.popen(r"C:\Tools\TD5\start_td5_adas_net.lnk")
            wait(30)
            if exists("1516275912874.png"):
                logger.info("TD5 started")
            else:
                logger.error('Unable to launch TD5')
        except Exception, e:
            str(e)
            logger.error(e)

    def tearDown(self):
        # After completion of the tests tear down exits the application
        try:
            if exists("1516275912874.png"):
                click(Pattern("1517330159528.png").targetOffset(19,-29))
                logger.info("Exiting TD5")
            else:
                logger.error('Unable to exit TD5')

        except Exception:
            print Exception

    def test_newProjectButton(self):
        try:
            logger.info("Checking new project button on toolbar")
            if exists("1516275912874.png"):
                click(Pattern("1517328688615.png").similar(0.80).targetOffset(-9,1))
                logger.info("Clicked on new button on toolbar")
                wait(2)
                if exists(Pattern("1517330266749.png").similar(0.90)):
                    logger.info("New project dialogue successfully started")
                    click("1517330523887.png")
                    assert True
                else:
                    logger.error("Unable to start New project import wizard")
                    assert False
            else:
                logger.error("Unable to click on new button in toolbar")

        except Exception, e:
            str(e)
            logger.error(e)

suite = unittest.TestLoader().loadTestsFromTestCase(Util)
outFile = open(r"C:\Tools\td5_gui_automation\Test_reports\toolbar_test.html", "w")
runner = HTMLTestRunner.HTMLTestRunner(stream=outFile, title='test report', description='Testing all toolbar icons on TD5 GUI' )
runner.run(suite)

HTML report:
It shows pass in the report and when i click pass it shows me this

pt1.1: Traceback (most recent call last):
  File "C:\Tools\td5_gui_automation\SikuliX\sikulix.jar\Lib\logging\__init__.py", line 874, in emit
    stream.write(fs % msg)
  File "C:\Tools\td5_gui_automation\SikuliX\sikulix.jar\Lib\logging\__init__.py", line 874, in emit
    stream.write(fs % msg)
ValueError: I/O operation on closed file
Logged from file ToolbarUtil.py, line 18
.... repeated

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
Vishal singh
Solved:
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said : 		#1

apparently you have a problem with the logger feature when running the scripts in the IDE.

Try to run your script from commandline.

BTW: your concept is a little bit weird:
You are using UnitTest/HTMLTestRunner, but there is not one assert, to report what's going wrong in a test, but instead you use the logging feature to report whats going wrong.

In setup you should simply exit, if the tests cannot be run, same in tearDown.

Revision history for this message
RaiMan (raimund-hocke) said : 		#2

... and there is some misunderstanding about tests pass/fail:
If in a test a not caught exception happens, the test is ended as error (neither pass nor fail).
If anything happens in the test, that does not break with an exception, the test will pass nevertheless.
Only assert yielding a result False will fail the test - but you do not have any asserts in your test!

Revision history for this message
Vishal singh (vishalsingh5900) said : 		#3

Thanks for the reply RaiMan
Can you show me an example like a code snippet what i am doing wrong exactly
I want to create test report as well as test log what i am doing after each step

Revision history for this message
RaiMan (raimund-hocke) said : 		#4

Generally you should think about using RobotFrameWork. if it is about testing with SikuliX.

There is a wrapping library:
https://github.com/rainmanwy/robotframework-SikuliLibrary

If you want to stick with your setup:
Use the SikuliX user logging - is much simpler:
http://sikulix-2014.readthedocs.io/en/latest/scripting.html#writing-and-redirecting-log-and-debug-messages

If I would do it (no logging):

setUp/tearDown should handle any bad situation and exit if it does not make sense to continue

   def setUp(self):
       TD5 = os.popen(r"C:\Tools\TD5\start_td5_adas_net.lnk")
       if not exists("TD5_ready.png", 30):
            print'Unable to launch TD5'
            exit(1) # makes no sense to continue

    def tearDown(self):
        if exists("TD5_ready.png"):
            click(Pattern("TD5_terminate.png").targetOffset(19,-29))
            if not waitVanish("TD5_ready.png", 30):
                print'Unable to stop TD5'
                exit(1) # makes no sense to continue
        else:
                print'TD5 not visible'
                exit(1) # makes no sense to continue

.... only one aspect per test

    def test_ClickNewProjectButton(self):
 #### see below
          if exists("TD5_ready.png"): ### not needed, setup has done it or exited!
                click(Pattern("TD5_newProjectButton.png").similar(0.80).targetOffset(-9,1))
                if exists(Pattern("1517330266749.png").similar(0.90), 5):
                    #### click("1517330523887.png") does not make sense here
                    assert True
                else:
                    assert False
#### this should never happen, see setup
            else:
                logger.error("Unable to click on new button in toolbar")

... so this would be my test:
    def test_ClickNewProjectButton(self):
        if exists(Pattern("TD5_newProjectButton").similar(0.80).targetOffset(-9,1))
            click()
            assert exists(Pattern("TD5_newProjectButtonResult").similar(0.90), 5), "TD5_newProjectButtonResult not there within 5 seconds)
       else:
           assert False, "TD5_newProjectButton not seen"

Revision history for this message
Vishal singh (vishalsingh5900) said : 		#5

Thank you so much for your help RaiMan
I think i got it
How to setup and how to use Unittest module
Thanks once again