How to generate reports using HtmlTestRunner for unittests which involve xml-rpc communication?

scaling means something like that? (more test def()' with XML-RPC calls)

def testx():
     assert getSomethingFromRemote(parmx) # XML_RPC call returning true/false

def testy():
     assert getSomethingFromRemote(parmy) # XML_RPC call returning true/false

or something like that? (more than one call in one def() )

def testz():
     x = getSomethingFromRemote(parmx) # XML_RPC call returning true/false
     y = getSomethingFromRemote(parmy) # XML_RPC call returning true/false
    assert x or y

Sorry for the lack of clarity in my question. I will re-frame it this way:

----------------------------------------------------------------------------------------------
On one machine ABC I have,

class ABC():
          def add():
                .......
          def subtract():
                .......

class MachineABCTest(unittest.TestCase):
         def test_ABC(self):
               x1 = abc_client.multiply() #method which resides in MachineXYZTest
               y1 = abc_client.divide() #method which resides in MachineXYZTest

suite = unittest.TestLoader().loadTestsFromTestCase(MachineABCTest)
self.abc_server.register_instance(ABC()) # methods add & subtract registered
runner = HTMLTestRunner.HTMLTestRunner(stream=fp, verbosity=2)
runner.run(suite)

--------------------------------------------------------------------------------------------------
And on the other machine,

class XYZ():
          def multiply():
                .......
          def divide():
                .......

class MachineXYZTest(unittest.TestCase):
         def test_XYZ(self):
              x2 = xyz_client.add() #method which resides in MachineXYZTest
              x2 = xyz_client.subtract() #method which resides in MachineXYZTest

suite = unittest.TestLoader().loadTestsFromTestCase(MachineXYZTest)
self.xyz_server.register_instance(XYZ()) # methods multiply & divide registered
runner = HTMLTestRunner.HTMLTestRunner(stream=fp, verbosity=2)
runner.run(suite)

----------------------------------------------------------------------------------------------

It would have generated report if there was only one call being made from ABC like
               x1 = abc_client.multiply() #method which resides in MachineXYZTest
and xyz_server serving that request. In this case the report will be generated on the client end(here, ABC).

The problem occurs if there's a 2-way communication occurring 'or' there are more than one request(as said above) being made(from ABC to XYZ or vice-versa) .

Thanks a lot.

Not clear, how this should work.

using self on indent level 0?
self.xyz_server.register_instance(XYZ()) # methods multiply & divide registered

what is self in this case?

In your snippet it is not clear when and how the servers are started. Since this always means, that the thread starting the server will wait forever, this thread would be blocked for other things besides handling requests.

Sorry Raiman, I thought of copying the relevant code only here.

Here, SimpleThreadedXMLRPCServer is being used. So there won't be any issue of the thread getting blocked. I am able to perform the xml-rpc two way communication without any issues. The problem is only with the generation of reports.

I'll expand the code snippet:
----------------------------------------------------------------------------------------------
On one machine ABC I have,

class ABC():
          def add():
                .......
          def subtract():
                .......

class MachineABCTest(unittest.TestCase):
         def test_ABC(self):
               x1 = abc_client.multiply() #method which resides in MachineXYZTest
               y1 = abc_client.divide() #method which resides in MachineXYZTest

suite = unittest.TestLoader().loadTestsFromTestCase(MachineABCTest)
runner = HTMLTestRunner.HTMLTestRunner(stream=fp, verbosity=2)
runner.run(suite)

class ServerThread(threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self)
        self.abc_server = SimpleThreadedXMLRPCServer(("XXX.XXX.XXX.1", 8000))
        self.abc_server.register_instance(ABC()) #methods add & subtract will get registered

    def run(self):
        self.moderator_srv.serve_forever()

abc_server = ServerThread()
abc_server.start()

--------------------------------------------------------------------------------------------------
And on the other machine,

class XYZ():
          def multiply():
                .......
          def divide():
                .......

class MachineXYZTest(unittest.TestCase):
         def test_XYZ(self):
              x2 = xyz_client.add() #method which resides in MachineXYZTest
              x2 = xyz_client.subtract() #method which resides in MachineXYZTest

suite = unittest.TestLoader().loadTestsFromTestCase(MachineXYZTest)
runner = HTMLTestRunner.HTMLTestRunner(stream=fp, verbosity=2)
runner.run(suite)

class ServerThread(threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self)
        self.xyz_server = SimpleThreadedXMLRPCServer(("XXX.XXX.XXX.2", 8000))
        self.xyz_server.register_instance(XYZ()) # methods multiply & divide will get registered

    def run(self):
        self.xyz_server.serve_forever()

xyz_server = ServerThread()
xyz_server.start()

----------------------------------------------------------------------------------------------

There's nothing wrong with the import statement as well

import HTMLTestRunner
reload(HTMLTestRunner)

dir = "D:\\Aravind\\ABC_XYZ_Test"
fp = file(os.path.join(dir, "TestResult.html"), "wb")

which I have included in both the files.

Thanks.

pls. do not blame me to be ignorant or foolish ;-)

... but taking the snippet the test suites are run before the server is started ???

-- you say: I am able to perform the xml-rpc two way communication without any issues?
... means outside any unittesting?

:-( Raiman, I won't ever do that...

Apologies....Actually what I've pasted here is just a template of what I have coded in the actual scenario.. So after I'm launching the unittests parallely form both the machines, there are other things to be performed before the actual xml-rpc communication takes place and I'm forced to put enough delays before the server is started. That's y I said "I am able to perform the xml-rpc two way communication without any issues". The issue is only with the report generation part.

Thanks a lot :)

But that exactly is, what I do not understand:

You are running the unit test suites well before you start the servers.
The unit tests contain XML-RPC client calls, that should fail, since the servers are not started yet ...!!??

Does not make sense for me.

I have coded in such a way that, by that time the xml-rpc requests are made, the servers will get started. Before that I have some other things to do using Sikuli scripts. And the xml-rpc communication does work out for me. That's why I said like that.

Ok, finally understood.

So this is one of your experiences:
- having only one XML_RPC call in the test case, lets the HTMLTestRunner produce a report
- if you add a second call to this test case, then HTMLTestrunner will not produce any report

... and this is true
- the test suite is run to the end in these cases
- there is no crash or other information about problems

How are you running your stuff? Using Sikuli-IDE or sikuli-script.jar from command line?

Thanks a lot...That's exactly what I meant.

I'm running it via the command line "Sikuli-IDE.bat \location\filename.sikuli

This is true as well.

- the test suite is run to the end in these cases.
- there is no crash or other information about problems.

might be a typo:

I think it should be:
Sikuli-IDE.bat -r \location\filename.sikuli

Yes.. it was a typo. All other things are working fine as expected except the report generation. I'm getting a 0KB Html report file in this case . But as I said above, reports are generated for test cases having only one XML_RPC call .
Thanks

Will look into during the next hours

I cannot reproduce your problem on 2 Macs.

- starting the servers on both machines
- running a client on each machine, that produces a series of popups on the other server machine (so I can simulate, that traffic is crossing, while both clients (HTMLTestRunner) are active).

this is what I used: https://dl.dropboxusercontent.com/u/42895525/xmlrpc-crosstest.zip

never at any time during my testing I got an empty report file.

Since your workflow logic sounds rather complex: are you sure, that you are not resetting the test runner logging somehow, so an empty report file is created in the end?

Raiman, thanks a lot but i regret to say, it's again not the scenario what I meant :-(

Here you are making only "192.168.0.100" as the server(xs) and multiple requests are made from the client(xc).
and this stuff works fine and generates report for me as well.
But what I want is, both machine ABC & machine XYZ should switch as client and server one after the another(not mandatory to be consecutive). Kindly have a look into Comment #4 code snippet...That's exactly why I am starting the servers on both the machines ABC & XYZ.

Thanks.

The code I gave you is only the snippet for one machine.

I have 4 processes running: a client and a server on both machines.
I did not make the effort, to have the server as a thread in the client process, but that should not make any difference.

I have:
Machine 1:
    Server 1
    Client 1

Machine 2:
    Server 2
    Client 2

Client 1 talks with Server 2 and
Client 2 talks with Server 1

The difference to your setup might be, that client and server on the same machine do not know anything about each other in terms of program scope. Which is only a problem, if you want some communication between the scheduled handlers and the clients on the same machine.

And here I come back to my question: If the client requests influence, what the client does on the other machine via the handler processing, than there might be a side effect, that resets the HTMLTestRunner somehow.

Ok.. understood.

I hope calling that displayABC is what you meant by "some communication between the scheduled handlers and the clients on the same machine". If so, Yes I need that communication on the same machine.

And can you please check out the following code snippet that I had been trying to run:

On Machine 1:

import math
import filecmp
import xmlrpclib
import os
import unittest
import SimpleXMLRPCServer, threading, SocketServer
from shutil import copyfile
import HTMLTestRunner
reload(HTMLTestRunner)

dir = "D:\MyWorkSpace\DellReportTest"
fp = file(os.path.join(dir, "TestResultDell.html"), "wb")

class SimpleThreadedXMLRPCServer(SocketServer.ThreadingMixIn, SimpleXMLRPCServer.SimpleXMLRPCServer):
    pass

class ABC():
    def displayABC(self):
        print "Inside Display ABC"
    def add(self):
        print "inside add"
        return True
    def subtract(self):
        print "inside subtract"
        return True

class ServerThread(threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self)
        self.abc_server = SimpleThreadedXMLRPCServer(("x.x.x.2", 8000))
        self.abc_server.register_instance(ABC()) #methods add & subtract will get registered

    def run(self):
        self.abc_server.serve_forever()

abc_server = ServerThread()
abc_server.start()

abc_client = xmlrpclib.ServerProxy("http://x.x.x.1:8000")

class MachineABCTest(unittest.TestCase):
    def setUp(self):
        pass
    def test_ABC(self):
        abc = ABC()
        abc.displayABC()
        x1 = abc_client.multiply() #method which resides in MachineXYZTest
        y1 = abc_client.divide() #method which resides in MachineXYZTest

    def tearDown(self):
        pass

suite = unittest.TestLoader().loadTestsFromTestCase(MachineABCTest)
runner = HTMLTestRunner.HTMLTestRunner(stream=fp, verbosity=2)
runner.run(suite)

On Machine 2:

from sikuli.Sikuli import *
import math
import filecmp
import xmlrpclib
import os
import unittest
import SimpleXMLRPCServer, threading, SocketServer
from shutil import copyfile
import HTMLTestRunner
reload(HTMLTestRunner)

dir = "D:\MyWorkSpace\HpReportTest"
fp = file(os.path.join(dir, "TestResultHp.html"), "wb")

class SimpleThreadedXMLRPCServer(SocketServer.ThreadingMixIn, SimpleXMLRPCServer.SimpleXMLRPCServer):
    pass

class XYZ():
    def displayXYZ(self):
        print "Inside Display XYZ"
    def multiply(self):
        print "inside multiply"
        return True
    def divide(self):
        print "inside divide"
        return True
class ServerThread(threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self)
        self.xyz_server = SimpleThreadedXMLRPCServer(("x.x.x.1", 8000))
        self.xyz_server.register_instance(XYZ()) # methods multiply & divide will get registered

    def run(self):
        self.xyz_server.serve_forever()

xyz_server = ServerThread()
xyz_server.start()

xyz_client = xmlrpclib.ServerProxy("http://x.x.x.2:8000")

class MachineXYZTest(unittest.TestCase):
    def setUp(self):
        pass
    def test_XYZ(self):
        xyz = XYZ()
        xyz.displayXYZ()
        x1 = xyz_client.add() #method which resides in MachineXYZTest
        y1 = xyz_client.subtract() #method which resides in MachineXYZTest

    def tearDown(self):
        pass

suite = unittest.TestLoader().loadTestsFromTestCase(MachineXYZTest)
runner = HTMLTestRunner.HTMLTestRunner(stream=fp, verbosity=2)
runner.run(suite)

So that I can get to know whether the HTMLTesRunner is getting reset anywhere. Thanks a lot !

Kindly insert a wait(10) after xyz_server.start() on either of the Machines in the above code and then run.

ie; (Here i'm adding it to the Machine 2 code)
xyz_server = ServerThread()
xyz_server.start()
wait(10)

so that you'll be able to see the two way communication in a sequential manner.

So by running the tests parallelly from both the machines, we'll get the output in this sequential manner:
inside multiply
inside divide
inside add
inside subtract
which means there's no issue in the two way communication , even though the server&client resides in the same file in both the machines.

But couldn't figure out what is stopping the report being generated.
How does the HtmlTestRunner get reset here, as you mentioned?

Thaaaanks a lot Raiman !!!

SimpleThreadedXMLRPCServer was preventing the report generation. Switched back to SimpleXMLRPCServer, in a thread, and it's working fine.

Currently I'm stopping the scripts using Ctrl+C, though it's not graceful :-(

Thanks RaiMan, that solved my question.

Since I always liked this XML-RPC as a good companion with Sikuli, I will put it on the list for possible extensions with version 1.1 and hope to get it running finally SimpleThreadedXMLRPCServer and HTMLTestrunner,which is another nice addition (like xldd, xlwt and more).

That will be really valuable. Thanks again.