Errors with typedefs in separate header

Asked by vldmrrr

Here is a simple example where the latest version of gccxmlparser fail to create accessors to structure members with simple types declared in separate header:
------------ mytypes.h ---------------------
#ifndef _MYTYPES_H_
#define _MYTYPES_H_

typedef unsigned char uint8;
typedef unsigned short uint16;
typedef unsigned int uint32;

#endif
-------------- base.h ------------------------
#include "mytypes.h"

#ifndef BASE_H_CONST
#define BASE_H_CONST

typedef struct
{
   int INT;
   uint32 U32;
   uint8 U8[32];
   uint16 U16;
} TST_STRUCT;

#endif
--------------- bind.py ------------------------
#!/usr/bin/env python
import os
import sys
import subprocess

import pybindgen.gccxmlparser

k_ModName = 'MyAPI'

k_api=['base.h']

def GenMY(path):
 includes = ['"%s"'%h for h in k_api]
 parser = pybindgen.gccxmlparser.ModuleParser(k_ModName)
 md = parser.parse(
  k_api,
  includes=includes)
 fname = os.path.join(path,k_ModName+'.c')
 f=file(fname,'w');
 md.generate(pybindgen.FileCodeSink(f))
 f.close()

if __name__ == '__main__':
 if len(sys.argv) != 2:
  print "Need path to MX installation as an only argument"
  sys.exit(1)
 GenMY(os.path.normpath(sys.argv[1]))
-------------------------------

Here is result of running of bind.py:
PYTHONPATH=~/dev/pybindgen-0.14.0 ./bind.py .
INFO Parsing source file "base.h" ...
INFO gccxml cmd: /usr/bin/gccxml -I"." "base.h" -fxml="/tmp/tmphxQb9m.xml"
INFO GCCXML version - 0.9 BUGGY
/mango/vbavdoni/tmp/tst/base.h:9: WrapperWarning: Return value '::uint32' error (used in TST_STRUCT::U32 [variable]): TypeLookupError(['uint32'],)
  uint32 U32;
/mango/vbavdoni/tmp/tst/base.h:10: WrapperWarning: Return value '::uint8[32]' error (used in TST_STRUCT::U8 [variable]): TypeLookupError(['uint8 [ 32 ]'],)
  uint8 U8[32];
/mango/vbavdoni/tmp/tst/base.h:11: WrapperWarning: Return value '::uint16' error (used in TST_STRUCT::U16 [variable]): TypeLookupError(['uint16'],)
  uint16 U16;
-------------------------------------

The generated code only have gettter and setter functions for INT member of TST_STRUCT, but not for any others. What is going wrong here?

Thanks,
Vladimir

Question information

Language:
English Edit question
Status:
Answered
For:
PyBindGen Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Gustavo Carneiro (gjc) said :
#1

I committed a fix to make 'unsigned short int' work.

uint8 U8[32] (array of char int) is simply not implemented in pybindgen, sorry.

The final problem is that the #include "mytypes.h" seems to confuse GCC-XML. If I copy-paste the typedefs into the main base.h header file then it works (except for the U8 field). I'll look more into this when I have more time.

Revision history for this message
vldmrrr (vldmrrr) said :
#2

Its not GCC-XML that is confused, it is more like me myself: the declarations from "mytypes.h" are filtered out by location_filter, because this file was not part of headers list passed to parser.parse. So after adding 'mytypes.h' to k_api list, and with your fix, I only get a single error for U8 array. How hard do you think it would be to add support for arrays? Maybe you know some workaround in current implementation?

BTW, does line 239 in inttype.py need the same fix in function UInt16Param as you did in line 200 in function UInt16Return?

Thanks,
Vladimir

Revision history for this message
vldmrrr (vldmrrr) said :
#3

Here is a proposed patch for you consideration. In addition of adding few more type aliases, it adds support of array members in structures.

Index: gccxmlparser.py
===================================================================
--- gccxmlparser.py (revision 4488)
+++ gccxmlparser.py (revision 4489)
@@ -268,7 +268,21 @@
             else:
                 warnings.warn("invalid annotation name %r" % name, AnnotationsWarning)

- cpp_type = normalize_name(type_info.partial_decl_string)
+ if hasattr(type_info,'partial_decl_string'):
+ decl = type_info.partial_decl_string
+ else:
+ decl = type_info
+ cpp_type = normalize_name(decl)
+
+ if cpp_type.endswith(']'):
+ ntype,c,l = cpp_type[:-1].rpartition('[')
+ if ntype.find('[') < 0:
+ ntype += '*';
+ l = int(l)
+ (cpp_type,), kwargs = self.lookup_return(ntype,annotations)
+ if kwargs.has_key('array_length'):
+ l = l*kwargs['array_length']
+ kwargs['array_length'] = l
         return (cpp_type,), kwargs

Index: typehandlers/base.py
===================================================================
--- typehandlers/base.py (revision 4488)
+++ typehandlers/base.py (revision 4489)
@@ -1401,7 +1401,15 @@
                 # existing.sort()
                 # raise TypeLookupError((tried_names, existing, self._type_aliases))
                 #else:
- raise TypeLookupError(tried_names)
+ rtypes = {}
+ for t in tried_names:
+ if self._type_aliases.has_key(t):
+ rtypes[t] = self._type_aliases[t]
+ elif self._type_aliases_rev.has_key(t):
+ rtypes[t] = "rev:%s"%self._type_aliases_rev[t]
+ else:
+ rtypes[t] = "???"
+ raise TypeLookupError(rtypes)
         else:
             logger.debug("try to lookup type handler for %r => success (%r)", name, rv)
             return rv
Index: typehandlers/inttype.py
===================================================================
--- typehandlers/inttype.py (revision 4488)
+++ typehandlers/inttype.py (revision 4489)
@@ -45,9 +45,9 @@
 class UnsignedIntPtrParam(PointerParameter):

     DIRECTIONS = [Parameter.DIRECTION_IN, Parameter.DIRECTION_OUT, Parameter.DIRECTION_INOUT]
- CTYPES = ['unsigned int*', 'uint32_t*']
+ CTYPES = ['uint32_t*', 'unsigned int*', 'unsigned long*', 'long unsigned*', 'long unsigned int*', 'unsigned long int*']

- def __init__(self, ctype, name, direction=Parameter.DIRECTION_IN, is_const=False,
+ def __init__(self, ctype, name, direction=Parameter.DIRECTION_INOUT, is_const=False,
                  default_value=None, transfer_ownership=False, array_length=None):
         super(UnsignedIntPtrParam, self).__init__(ctype, name, direction, is_const, default_value, transfer_ownership)
         self.array_length = array_length
@@ -146,7 +146,7 @@
                   Parameter.DIRECTION_IN|Parameter.DIRECTION_OUT]
     CTYPES = ['int*']

- def __init__(self, ctype, name, direction=None, is_const=None, transfer_ownership=None):
+ def __init__(self, ctype, name, direction=Parameter.DIRECTION_INOUT, is_const=None, transfer_ownership=None):
         if direction is None:
             if is_const:
                 direction = Parameter.DIRECTION_IN
@@ -197,7 +197,7 @@

 class UInt16Return(ReturnValue):

- CTYPES = ['uint16_t', 'unsigned short', 'unsigned short int']
+ CTYPES = ['uint16_t', 'unsigned short', 'unsigned short int', 'short unsigned int' ]

     def get_c_error_return(self):
         return "return 0;"
@@ -236,7 +236,7 @@
 class UInt16Param(Parameter):

     DIRECTIONS = [Parameter.DIRECTION_IN]
- CTYPES = ['uint16_t', 'unsigned short', 'unsigned short int']
+ CTYPES = ['uint16_t', 'unsigned short', 'unsigned short int', 'short unsigned int' ]

     def convert_c_to_python(self, wrapper):
         assert isinstance(wrapper, ReverseWrapperBase)
@@ -250,6 +250,77 @@
                                               'PyErr_SetString(PyExc_ValueError, "Out of range");')
         wrapper.call_params.append(name)

+class UInt16PtrParam(PointerParameter):
+
+ DIRECTIONS = [Parameter.DIRECTION_IN, Parameter.DIRECTION_OUT, Parameter.DIRECTION_INOUT]
+ CTYPES = ['uint16_t*', 'unsigned short*', 'unsigned short int*', 'short unsigned int*' ]
+
+ def __init__(self, ctype, name, direction=Parameter.DIRECTION_INOUT, is_const=False,
+ default_value=None, transfer_ownership=False, array_length=None):
+ super(UInt16PtrParam, self).__init__(ctype, name, direction, is_const, default_value, transfer_ownership)
+ self.array_length = array_length
+ if transfer_ownership:
+ raise NotSupportedError("%s: transfer_ownership=True not yet implemented." % ctype)
+
+ def convert_c_to_python(self, wrapper):
+ if self.direction & self.DIRECTION_IN:
+ wrapper.build_params.add_parameter('i', ['*'+self.value])
+
+ if self.direction & self.DIRECTION_OUT:
+ wrapper.parse_params.add_parameter('i', [self.value], self.name)
+
+ def convert_python_to_c(self, wrapper):
+ #assert self.ctype == 'unsigned int*'
+ if self.array_length is None:
+ name = wrapper.declarations.declare_variable(str(self.type_traits.target), self.name)
+ wrapper.call_params.append('&'+name)
+ if self.direction & self.DIRECTION_IN:
+ wrapper.parse_params.add_parameter('i', ['&'+name], self.name)
+ if self.direction & self.DIRECTION_OUT:
+ wrapper.build_params.add_parameter('i', [name])
+
+ else: # complicated code path to deal with arrays...
+
+ name = wrapper.declarations.declare_variable(str(self.type_traits.target), self.name, array="[%i]" % self.array_length)
+ py_list = wrapper.declarations.declare_variable("PyObject*", "py_list")
+ idx = wrapper.declarations.declare_variable("int", "idx")
+ wrapper.call_params.append(name)
+ if self.direction & self.DIRECTION_IN:
+ elem = wrapper.declarations.declare_variable("PyObject*", "element")
+ wrapper.parse_params.add_parameter('O!', ['&PyList_Type', '&'+py_list], self.name)
+ wrapper.before_call.write_error_check(
+ 'PyList_Size(%s) != %i' % (py_list, self.array_length),
+ 'PyErr_SetString(PyExc_TypeError, "Parameter `%s\' must be a list of %i ints/longs");'
+ % (self.name, self.array_length))
+
+ wrapper.before_call.write_code(
+ "for (%s = 0; %s < %i; %s++) {" % (idx, idx, self.array_length, idx))
+ wrapper.before_call.indent()
+
+ wrapper.before_call.write_code("%(elem)s = PyList_GET_ITEM(%(py_list)s, %(idx)s);" % vars())
+ wrapper.before_call.write_error_check(
+ '!(PyInt_Check(%(elem)s) || PyLong_Check(%(elem)s))',
+ 'PyErr_SetString(PyExc_TypeError, "Parameter `%s\' must be a list of %i ints / longs");'
+ % (self.name, self.array_length))
+ wrapper.before_call.write_code("%(name)s[%(idx)s] = (uint16_t) PyLong_AsUnsignedInt(%(elem)s);" % vars())
+
+ wrapper.before_call.unindent()
+ wrapper.before_call.write_code('}')
+
+ if self.direction & self.DIRECTION_OUT:
+ wrapper.after_call.write_code("%s = PyList_New(%i);" % (py_list, self.array_length))
+
+ wrapper.after_call.write_code(
+ "for (%s = 0; %s < %i; %s++) {" % (idx, idx, self.array_length, idx))
+ wrapper.after_call.indent()
+ wrapper.after_call.write_code("PyList_SET_ITEM(%(py_list)s, %(idx)s, PyLong_FromUnsignedLong(%(name)s[%(idx)s]));"
+ % vars())
+ wrapper.after_call.unindent()
+ wrapper.after_call.write_code('}')
+
+ wrapper.build_params.add_parameter("N", [py_list])
+
+
 class Int16Param(Parameter):

     DIRECTIONS = [Parameter.DIRECTION_IN]
@@ -292,7 +363,7 @@
 class UInt8Param(Parameter):

     DIRECTIONS = [Parameter.DIRECTION_IN]
- CTYPES = ['uint8_t']
+ CTYPES = ['uint8_t', 'unsigned char']

     def convert_c_to_python(self, wrapper):
         assert isinstance(wrapper, ReverseWrapperBase)
@@ -309,7 +380,7 @@

 class UInt8Return(ReturnValue):

- CTYPES = ['uint8_t']
+ CTYPES = ['uint8_t', 'unsigned char']

     def get_c_error_return(self):
         return "return 0;"
@@ -325,7 +396,42 @@
     def convert_c_to_python(self, wrapper):
         wrapper.build_params.add_parameter("i", ['(int)' + self.value], prepend=True)

+class UInt8PtrReturn(PointerReturnValue):

+ DIRECTIONS = [Parameter.DIRECTION_IN, Parameter.DIRECTION_OUT,
+ Parameter.DIRECTION_IN|Parameter.DIRECTION_OUT]
+ CTYPES = ['uint8_t*','unsigned char *']
+
+ def __init__(self, ctype,
+ is_const=None, default_value=None, transfer_ownership=None,
+ array_length=None):
+ self.array_length = array_length
+ if is_const:
+ direction = Parameter.DIRECTION_IN
+
+ super(UInt8PtrReturn, self).__init__(ctype, is_const)
+
+ def convert_python_to_c(self, wrapper):
+ name = wrapper.declarations.declare_variable('uint8_t*',"tmp")
+ if self.array_length is None:
+ wrapper.parse_params.add_parameter("B", [name])
+ else:
+ wrapper.parse_params.add_parameter(
+ '['+'B'*self.array_length+']',
+ ['%s+%d'%(self.value,i) for i in range(self.array_length)],
+ prepend=True
+ )
+
+ def convert_c_to_python(self, wrapper):
+ if self.array_length is None:
+ wrapper.build_params.add_parameter("B", [self.value], self.name)
+ else:
+ wrapper.build_params.add_parameter(
+ '['+'B'*self.array_length+']',
+ ['%s[%d]'%(self.value,i) for i in range(self.array_length)],
+ prepend=True
+ )
+
 class UnsignedLongLongParam(Parameter):

     DIRECTIONS = [Parameter.DIRECTION_IN]
@@ -369,7 +475,7 @@
 class UnsignedLongParam(Parameter):

     DIRECTIONS = [Parameter.DIRECTION_IN]
- CTYPES = ['unsigned long']
+ CTYPES = ['unsigned long', 'long unsigned', 'long unsigned int', 'unsigned long int']

     def get_ctype_without_ref(self):
         return str(self.type_traits.ctype_no_const)
@@ -394,7 +500,7 @@

 class UnsignedLongReturn(ReturnValue):

- CTYPES = ['unsigned long', 'long unsigned int']
+ CTYPES = ['unsigned long', 'long unsigned', 'long unsigned int', 'unsigned long int']

     def get_c_error_return(self):
         return "return 0;"
@@ -493,7 +599,10 @@
                   Parameter.DIRECTION_IN|Parameter.DIRECTION_OUT]
     CTYPES = ['int8_t*']

- def __init__(self, ctype, name, direction=None, is_const=None, default_value=None, transfer_ownership=None):
+ def __init__(self, ctype, name, direction=Parameter.DIRECTION_INOUT,
+ is_const=None, default_value=None, transfer_ownership=None,
+ array_length=None):
+ self.array_length = array_length
         if direction is None:
             if is_const:
                 direction = Parameter.DIRECTION_IN
@@ -520,9 +629,12 @@

     DIRECTIONS = [Parameter.DIRECTION_IN, Parameter.DIRECTION_OUT,
                   Parameter.DIRECTION_IN|Parameter.DIRECTION_OUT]
- CTYPES = ['uint8_t*']
+ CTYPES = ['uint8_t*','unsigned char *']

- def __init__(self, ctype, name, direction=None, is_const=None, default_value=None, transfer_ownership=None):
+ def __init__(self, ctype, name, direction=Parameter.DIRECTION_INOUT,
+ is_const=None, default_value=None, transfer_ownership=None,
+ array_length=None):
+ self.array_length = array_length
         if direction is None:
             if is_const:
                 direction = Parameter.DIRECTION_IN

Revision history for this message
Gustavo Carneiro (gjc) said :
#4

Thank you. I've been a little busy later, but your patch will be looked at eventually...

Revision history for this message
Gustavo Carneiro (gjc) said :
#5

vldmr, your patch posted inline seems to have lost proper indentation. Could you please open a bug report instead and attach the patch file separately?

Can you help with this problem?

Provide an answer of your own, or ask vldmrrr for more information if necessary.

To post a message you must log in.