PyBindGen

Raw memory access

Asked by Morten V. Pedersen

Hi I would like to pass some binary data to my c++ API, so basically wrap a function e.g. void rawMemoryAccess(char *data, int length);

I cannot seem to figure out how to do this, in particular I want the generated PyArg_ParseTupleAndKeywords to use the "#s" and not "s". So that we can pass a Python string which may contain embedded null bytes.

Any clues :)

Best regards,
M

Edit: Maybe the underpinning question here is how to handle and c-style array.

Question information

Language:
English Edit question
Status:
Solved
For:
PyBindGen Edit question
Assignee:
No assignee Edit question
Solved by:
Gustavo Carneiro
Solved:
Last query:
Last reply:
Revision history for this message
Morten V. Pedersen (morten-mortenvp) said : 		#1

edit

Revision history for this message
Best Gustavo Carneiro (gjc) said : 		#2

You should make a new type handler, possibly based on pybindgen.typehandlers.string.StdStringParam. For instance (untested code)

class RawMemParam(Parameter):

    DIRECTIONS = [Parameter.DIRECTION_IN]
    CTYPES = []

    def convert_c_to_python(self, wrapper):
        raise NotImplementedError

    def convert_python_to_c(self, wrapper):
        name = wrapper.declarations.declare_variable("const char *", self.name)
        name_len = wrapper.declarations.declare_variable("Py_ssize_t", self.name+'_len')
        wrapper.parse_params.add_parameter('s#', ['&'+name, '&'+name_len], self.value)
        wrapper.call_params.append(name)
        wrapper.call_params.append('(int) ' + name_len)

You then use this type by directly instantiating the class:

   mod.add_function('rawMemoryAccess', 'void', ['RawMemParam 'char*', 'data')])

This only works for Python to C++ wrappers, the reverse wrappers (for virtual methods) need more code and are more complicated.

Revision history for this message
Morten V. Pedersen (morten-mortenvp) said : 		#3

Thanks Gustavo Carneiro, that solved my question.

Revision history for this message
Morten V. Pedersen (morten-mortenvp) said : 		#4

I'm thinking whether it would make sense to make #s the default for char*/const char*. Just using a dummy int for the length. In that case both zero terminated strings and raw binary data would be handled correctly (with a minor inefficiency of the dummy int). Or what do you think?

Best Morten

Revision history for this message
Morten V. Pedersen (morten-mortenvp) said : 		#5

One more question :)
Is there a way to replace the standard type handlers. So in this case if I want to use my own char* type handler instead of the pybindgen supplied one?

I tried:
from pybindgen.typehandlers import base as typehandlers
typehandlers.param_type_matcher.register('char *', RawMemParam)
but had to do a:
name = ctypeparser.normalize_type_string("char*")
typehandlers.param_type_matcher._types[name] = RawMemParam

Maybe an API could be added to replace type handlers.

Revision history for this message
Gustavo Carneiro (gjc) said : 		#6

1. I don't see the point in using s# instead of s everytime. With s#, I don't see anything in the documentation that says the char* pointer will point to a null-terminated string, what if the string is not null terminated, but the wrapped C function expects null-terminated string?

2. I guess we can use an API for overriding a registered type handler. Feel free to open a bug report if you want it.

Revision history for this message
Morten V. Pedersen (morten-mortenvp) said : 		#7

Good point, default s# would not be good for everybody.

Btw. thanks for a really nice tool - I manged to get my library wrapped pretty easily :)

Best regards,
Morten