Set Execution Ordinal for Test Set via Command Line

Asked by Seumas MacRae

We have a defined Smoke Test Set established and it is set to run each day after a new nightly build is deployed. This execution is being launched via a command line script. Due to pre-requisites that some of our newer test assets have, how can I set the execution ordinal for the contents of the Test Set folders? Currently, the execution appears to happen by random selection.

Thanks,
Seumas

Question information

Language:
English Edit question
Status:
Answered
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Mike (maestro+++) said :
#1

Just for everybody's benefit, what do you mean by 'test ordinal' and what do you mean by' execution appears to happen by random selection'?

Thanks

Revision history for this message
Seumas MacRae (djspartacus) said :
#2

Test Ordinal = set the order in which the test scripts should be run/executed. For example, the file structure looks like this:

TestSets
  ApplicationName
    BVT
      CloseApp1.sikuli
      CreateRulesTest.sikuli
      DeleteRulesTest.sikuli
      OpenApp1.sikuli

I'd like to be able to tell Sikuli to execute/run the tests in the following order:
1) OpenApp1.sikuli
2) CreateRulesTest.sikuli
3) DeleteRulesTest.sikuli
4) CloseApp1.sikuli

As for the "random selection" question: when I watch Sikuli run the tests from the command line script, I've noticed that it doesn't always run them in the same ordinal/order. So, it appears that it randomly picks which one it is going to launch during test execution. Since we are making our scripts more modular/portable, I need to be able to specify at runtime the running order (ordinal) I want Sikuli to follow.

Cheers,
Seumas

Revision history for this message
RaiMan (raimund-hocke) said :
#3
Revision history for this message
TestMechanic (ndinev) said :
#4

If you add code sample that demonstrates how you are running test maybe we can help.

I am using unittest module and organizing tests as methods of a class. The keep test ordered methods looks like

def test_001_
def test_002_

Then when tests are loaded for execution the are always sorted by name

Can you help with this problem?

Provide an answer of your own, or ask Seumas MacRae for more information if necessary.

To post a message you must log in.