Error some soap clients

Asked by Hakan on 2011-08-18

Hi;

I want to use this application, and so I tried. I started working with the new Python, so I am not yet fully dominated the subject. However, there is a problem and I would like to consult you.

The problem is, Suds is not really a problem when I try using the client. However, returning the following error when I try to, for example soapUI. I wonder what could be the reason for this? I searched a bit but I could not find the cause, so we wanted to consult you. I hope this is never an issue for you.

RETURNED:
--------------------------------------------------
Traceback (most recent call last):
 File "/usr/lib/python2.6/site-packages/ladon-0.6.1-py2.6.egg/ladon/server/wsgi_application.py", line 356, in __call__
   response_part = dispatcher.dispatch_request(request_data,environ)
 File "/usr/lib/python2.6/site-packages/ladon-0.6.1-py2.6.egg/ladon/server/dispatcher.py", line 148, in dispatch_request
   req_dict = self.iface.parse_request(request_data,encoding=self.response_encoding)
 File "/usr/lib/python2.6/site-packages/ladon-0.6.1-py2.6.egg/ladon/interfaces/base.py", line 54, in parse_request
   return self._request_handler.parse_request(soap_body,self._sinfo,encoding)
 File "/usr/lib/python2.6/site-packages/ladon-0.6.1-py2.6.egg/ladon/interfaces/soap.py", line 367, in parse_request
   m = re.match("^ns(\d+)$",soap_methodprefix)
 File "/usr/lib64/python2.6/re.py", line 137, in match
   return _compile(pattern, flags).match(string)
TypeError: expected string or buffer
--------------------------------------------------

Question information

Language:
English Edit question
Status:
Solved
For:
ladon Edit question
Assignee:
No assignee Edit question
Last query:
2011-08-19
Last reply:
2012-01-08
jsgaarde (jakob-simon-gaarde) said : #1

Hi Hakan.

There is a Ladon user named Romankrv who has reported the precise same error: https://answers.launchpad.net/ladon/+question/166247

The problem is actually not in Ladon but in soapUI that doesn't honer the SOAP 1.1 specification.

I might be able to make Ladon tolerant to the problem. But I don't like to hack good software to make broken software work. However if you send me the raw SOAP request message that causes the exception, I will make a patch for you to apply on your local installation.

Best Regards
Jakob Simon-Gaarde

Hakan (onebemail) said : #2

Hi;

I understand. but, soapUI web site says:

"soapUI supports the most widely used version of WSDL 1.1 bindings for SOAP 1.1 and 1.2 and Corresponding. The now-obsolete SOAP-Encoding standard (used for specifying the XML before XML Schema was formalized contracts) is partially supported."

In addition, a similar error when I tried going on with Eclipse and soamoa.

For example, eclipse show following error:

"An unexpected error has occurred.
500
An Error occured while processing the request"

soapUI instance you want to request it by:

RAW:
----------
HTTP/1.1 POST http://ip_address/project/OperatingSystem/soap
Accept-Encoding: gzip, deflate
Content-Type: text / xml; charset = UTF-8
SOAPAction: "http://ip_address:80/project/OperatingSystem/soap/getHostName"
User-Agent: Jakarta Commons-HttpClient/3.1
Host: ip_address
Content-Length: 381

XML:
----------
<soapenv: Envelope xmlns: xsi =" http://www.w3.org/2001/XMLSchema-instance "xmlns: xsd =" http://www.w3.org/2001/XMLSchema "xmlns: soapenv =" http://schemas.xmlsoap.org/soap/envelope/ "xmlns: urn =" urn: OperatingSystem ">
    <soapenv:Header/>
    <soapenv:Body>
       <urn:getHostName soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"/>
    </ soapenv: Body>
</ soapenv: Envelope>

Additional, these types of cases, the error does not appear in python? Seems an appropriate message?

I give this example to understand the way this topic.

Thanks for your interest
Best Regards

jsgaarde (jakob-simon-gaarde) said : #3

Hi Hakan.

The XML above can't even be parsed as XML, and that is probably because you are just pasting it into the answer-thread. I really want to help you, and I have time to do it today, but you need to send the SOAP request so it can be parsed (use http://pastebin.com)

Best Regards
Jakob Simon-Gaarde

jsgaarde (jakob-simon-gaarde) said : #4

BTW - please tell me where the request is from, samoa or soapUI. I'd like to se requests from both :-)

Hakan (onebemail) said : #5

Hi;

Above raw log, soapUI request. And following the log, raw answer copied from SoapUI.

RAW ANSWER FROM LADON. COPIED FROM SOAPUI:
--------------------------------------------------
HTTP/1.1 500 An Error occured while processing the request
Date: Fri, 19 Aug 2011 08:14:19 GMT
Server: Apache/2.2.17 (FreeBSD) mod_ssl/2.2.17 OpenSSL/0.9.8q DAV/2 mod_wsgi/3.3 Python/2.6.6
Content-Length: 964
Connection: close
Content-Type: text/plain; charset=UTF-8

Traceback (most recent call last):
  File "/usr/local/lib/python2.6/site-packages/ladon-0.6.1-py2.6.egg/ladon/server/wsgi_application.py", line 356, in __call__
    response_part = dispatcher.dispatch_request(request_data,environ)
  File "/usr/local/lib/python2.6/site-packages/ladon-0.6.1-py2.6.egg/ladon/server/dispatcher.py", line 148, in dispatch_request
    req_dict = self.iface.parse_request(request_data,encoding=self.response_encoding)
  File "/usr/local/lib/python2.6/site-packages/ladon-0.6.1-py2.6.egg/ladon/interfaces/base.py", line 54, in parse_request
    return self._request_handler.parse_request(soap_body,self._sinfo,encoding)
  File "/usr/local/lib/python2.6/site-packages/ladon-0.6.1-py2.6.egg/ladon/interfaces/soap.py", line 367, in parse_request
    m = re.match("^ns(\d+)$",soap_methodprefix)
  File "/usr/local/lib/python2.6/re.py", line 137, in match
    return _compile(pattern, flags).match(string)
TypeError: expected string or buffer
--------------------------------------------------

Thank you for your very interest
Best Regards

jsgaarde (jakob-simon-gaarde) said : #6

Hi Hakan.

You need to send the request using http://pastebin.com , otherwise I can't use it, cause this question/answer system does not show conserve the raw request. If I copy the "RAW" text from your message above it _cannot_ be parsed as xml, you can try yourself or trust me ;-) Use pastebin.com that's the normal way of communicating data in this system.

Best regards
Jakob Simon-Gaarde

jsgaarde (jakob-simon-gaarde) said : #7

I don't need the exception I need the SOAP XML request.

Hakan (onebemail) said : #8

Hi;

I am sorry, i did not see. The url you want: http://pastebin.com/M84ZNjyr

This right?

Thanks,
Best Regards

jsgaarde (jakob-simon-gaarde) said : #9

Much better!
tune-in later ;-)

jsgaarde (jakob-simon-gaarde) said : #10

Hi Hakan.

I have looked at the request - it seems that the only problem is that soapUI prettyfies XML. Ladon is expecting unprettyfied XML.

I will probably commit a fix that will unprettify incomming xml. Meanwhile - you can theck if SoapUI can be setup to send raw XML requests (meaning: non-prettyfied XML)

Best Regards
Jakob Simon-Gaarde

Hakan (onebemail) said : #11

Hi;

Maybe prettyprint is not the problem. Because, i traced the traffic with ngrep from server side. The incoming traffic from soapui not contain newline or tab(example in the following). This raw data printing a text file, and checked from file.

I have not send the file, because can not attach a files here.

Can you update ladon this issue?

Thanks,
Best Regards

jsgaarde (jakob-simon-gaarde) said : #12

With all do respect, I think you are recieving it prettyfied - study this:

http://pastebin.com/ghN194BR

Now I have to ask you if you have post-altered the raw request message xml after pasting it, if not SoapUI is also producing incorrect content-length headers - which is far more serious cause some webservers won't let you "over-read" for security reasons (buffer-overload attacks). For instance if your client sends a message body which has the actual byte-length of 377, and your client sends an incorrect content-length 381. Then the webserver will prevent the Ladon Webservice application object from reading and effectively kill the executing thread.

Best Regards
Jakob Simon-Gaarde

Hakan (onebemail) said : #13

Hi;

I checked for your idea. Data size, exactly as he wrote in the header(and yes, sending pretty print lines true). But:
I sent to you by email, received dump file with tcpdump. If examine you see, the correct size.

Thank you for very interest,
Best Regards

gthomas (gt-launchpad) said : #14

Hi,

For first thought, I'd blame soap_methodprefix (is None). And indeed it is, but I just don't understand why is the prettyfying whitespace treated as TEXTNODE by minidom?

This solves the problem, but still doesn't know why's there a TEXT_NODE!?!

=== modified file 'frameworks/python/src/ladon/interfaces/soap.py'
--- /tmp/bzr-diff-1UaNdB/old/frameworks/python/src/ladon/interfaces/soap.py 2011-06-08 00:23:08.783999000 +0200
+++ /tmp/trunk/frameworks/python/src/ladon/interfaces/soap.py 2011-08-19 22:24:15.986301288 +0200
@@ -335,6 +335,8 @@
   soap_body = soap_envelope.getElementsByTagNameNS('*','Body')[0]
   soap_method = soap_body.childNodes[0]
   soap_methodprefix = soap_method.prefix
+ soap_method = (node for node in soap_body.childNodes
+ if node.nodeType == node.ELEMENT_NODE).next()
   m = re.match("^ns(\d+)$",soap_methodprefix)
   servicenumber = None
   if m: servicenumber = int(m.groups()[0])

gthomas (gt-launchpad) said : #15

Sorry, the corrected patch:

=== modified file 'frameworks/python/src/ladon/interfaces/soap.py'
--- /tmp/bzr-diff-ORQS7a/old/frameworks/python/src/ladon/interfaces/soap.py 2011-06-08 00:23:08.783999000 +0200
+++ /tmp/trunk/frameworks/python/src/ladon/interfaces/soap.py 2011-08-19 22:27:44.876301336 +0200
@@ -333,7 +333,8 @@
   doc = md.parseString(soap_body)
   soap_envelope = doc.getElementsByTagNameNS('*','Envelope')[0]
   soap_body = soap_envelope.getElementsByTagNameNS('*','Body')[0]
- soap_method = soap_body.childNodes[0]
+ soap_method = (node for node in soap_body.childNodes
+ if node.nodeType == node.ELEMENT_NODE).next()
   soap_methodprefix = soap_method.prefix
   m = re.match("^ns(\d+)$",soap_methodprefix)
   servicenumber = None

jsgaarde (jakob-simon-gaarde) said : #16

Hi gthomas.

Yes, that solves the problem for extracting the method name. It would be nice if someone could create a more complex service ie. involvong ie a list of LadonType as argument and as results to see if prettyfied request messages are handled OK. Could you do that gthomas?

Best regards
Jakob Simon-Gaarde

jsgaarde (jakob-simon-gaarde) said : #18

Nice gthomas!

I'll try and test your patch with nested types, to be sure it works with recursion in function nodeListToDict.

Best regards
Jakob Simon-Gaarde

jsgaarde (jakob-simon-gaarde) said : #19

BTW: why did you test it with Ladon version 0.5.2 and not 0.6.1?

/ Jakob

jsgaarde (jakob-simon-gaarde) said : #20

Hi gthomas.

First of all thanks for your contribution, I welcome all the help I can get. If at some point you would like to join in (after a couple of contributions) goto https://launchpad.net/~ladon-dev-team and propose yourself :-)

I am currently testing your changes with a service taking nested types. I saw you proposed merging, but it doesn't make much sense to merge onto an old version of Ladon, so could I ask you to branch https://code.launchpad.net/~ladon-dev-team/ladon/ladon when you propose a fix :-)

Again thanks for helping!

/ Jakob

jsgaarde (jakob-simon-gaarde) said : #21

Hi gthomas.

Sorry for not answering before now - family stuff took my time :-)

Your fix worked as long as SOAP requests didn't recursively parse through nested LadonTypes. And to solve that problem it was actually only nessecary to copy your generator solution to the recursive call in the nodeListToDict function. Unfortunately while testing your contribution I stepped on a small landmine, and found an all together different bug. When defining arguments of LadonTypes that had attributes of lists of LadonTypes, the list part would be entirely ignored. ie beneeth where the argument glist (LadonType: GroceryList) references a list of LadonType Purchase:

<?xml version="1.0" ?>
<SOAP-ENV:Envelope SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns0="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="http://www.w3.org/2001/XMLSchema" xmlns:ns2="http://schemas.xmlsoap.org/soap/encoding/" xmlns:ns3="urn:ShopService" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <SOAP-ENV:Header/>
  <ns0:Body>
    <ns3:addToBasket>
      <glist xsi:type="ns3:GroceryList">
        <purchases ns2:arrayType="ns3:Purchase[2]" xsi:type="ns3:ArrayOfPurchase">
          <item xsi:type="ns3:Purchase">
            <grocery xsi:type="ns3:Grocery">
              <itemname xsi:type="ns1:string">
                item1
              </itemname>
              <price xsi:type="ns1:decimal">
                235.54
              </price>
            </grocery>
            <quantity xsi:type="ns1:long">
              2
            </quantity>
          </item>
          <item xsi:type="ns3:Purchase">
            <grocery xsi:type="ns3:Grocery">
              <itemname xsi:type="ns1:string">
                item2
              </itemname>
              <price xsi:type="ns1:decimal">
                354.3
              </price>
            </grocery>
            <quantity xsi:type="ns1:long">
              10
            </quantity>
          </item>
        </purchases>
      </glist>
    </ns3:addToBasket>
  </ns0:Body>
</SOAP-ENV:Envelope>

Anyway - I decieded to rewrite the whole requesthandler using the SAX parser instead of DOM.

Best Regards
Jakob Simon-Gaarde

P.S. May still be a small problem when using lists of primitives, but thats an easy fix tomorrow :-)

jsgaarde (jakob-simon-gaarde) said : #22

Changes on the repository:
http://bazaar.launchpad.net/~ladon-dev-team/ladon/ladon/revision/41

This of course also fixes the original question of this thread.

/ Jakob

gthomas (gt-launchpad) said : #23

Thanks the fix!

I know the (family) problem, I have one, too. And now the workdays came, so I'll be busy, respond spacely... - so I don't know when will I be able to test these nice improvements.

GThomas

jsgaarde (jakob-simon-gaarde) said : #24

Question asker GThomas - accepted a fix