Newbie question - getting started

Asked by Johan Råde

I have a C++ library whose interface consists of tree classes: TextTable, TextRow and NumpyArrayBuilder.
These classes are declared in three header files: TextTable.h, TextRow.h and NumpyArrayBuilder.h.
I want to build a Python extension module called TextTable that wraps these three classes.
I understand the script to generate the Python C-API code should look something like:

    import sys
    import pybindgen
    from pybindgen import FileCodeSink
    from pybindgen.gccxmlparser import ModuleParser

    module_parser = ModuleParser(...)
    module = module_parser.parse(...)
    module.add_include(...)

    pybindgen.write_preamble(FileCodeSink("PythonWrapper.cpp"))
    module.generate(FileCodeSink("PythonWrapper.cpp"))

What should the ... be?

Question information

Language:
English Edit question
Status:
Solved
For:
PyBindGen Edit question
Assignee:
No assignee Edit question
Solved by:
Johan Råde
Solved:
Last query:
Last reply:
Revision history for this message
Johan Råde (johan-rade) said :
#1

The following code

    import pybindgen
    from pybindgen.gccxmlparser import ModuleParser

    module_parser = ModuleParser('TextTable')
    module = module_parser.parse(['TextTable.h'])

results in the error message

1> Traceback (most recent call last):
1> File "TextTablePyBindGen.py", line 7, in <module>
1> module = module_parser.parse(['TextTable.h'])
1> File "C:\Python27\pybindgen\gccxmlparser.py", line 586, in parse
1> pygen_classifier, gccxml_options)
1> File "C:\Python27\pybindgen\gccxmlparser.py", line 683, in parse_init
1> self.declarations = parser.parse(header_files, self.gccxml_config)
1> File "C:\Python27\lib\site-packages\pygccxml\parser\__init__.py", line 50, in parse
1> answer = parser.read_files(files, compilation_mode)
1> File "C:\Python27\lib\site-packages\pygccxml\parser\project_reader.py", line 225, in read_files
1> return self.__parse_file_by_file(files)
1> File "C:\Python27\lib\site-packages\pygccxml\parser\project_reader.py", line 246, in __parse_file_by_file
1> , self.__decl_factory )
1> File "C:\Python27\lib\site-packages\pygccxml\parser\source_reader.py", line 87, in __init__
1> self.__config.raise_on_wrong_settings()
1> File "C:\Python27\lib\site-packages\pygccxml\parser\config.py", line 189, in raise_on_wrong_settings
1> raise RuntimeError( msg )
1> RuntimeError: gccxml_path("") should exists or to be a valid file name.

What am I doing wrong?
I'm using PyBindGen 0.15.0 on Windows XP.

Revision history for this message
Gustavo Carneiro (gjc) said :
#2

Have you installed gccxml? Try modifying the PATH environment variable to make sure that gccxml.exe is found in the PATH.

Revision history for this message
Johan Råde (johan-rade) said :
#3

Thank you, that fixed that problem. I had installed gccxml but I had not set the PATH variable.

I have a header file StdAfx.h that contains lots of stuff like "#include <fstream>" that must be included before TextTable.h can be parsed. How do I take care of that?

Revision history for this message
Gustavo Carneiro (gjc) said :
#4

    module.add_include('<fstream>')

Revision history for this message
Johan Råde (johan-rade) said :
#5

The script

    import pybindgen
    f rom pybindgen import FileCodeSink
    from pybindgen.gccxmlparser import ModuleParser

    module_parser = ModuleParser('TextTable')
    module = module_parser.parse(['../TextTable/TextRow.h'])

gives the error message

1> INFO Parsing source file "../TextTable/TextRow.h" ...
1> INFO gccxml cmd: ""C:\Program Files\gccxml 0.9\bin\gccxml.exe" -I"." "../TextTable/TextRow.h" -fxml="c:\docume~1\rade\locals~1\temp\tmpz7eizs.xml""
1> Traceback (most recent call last):
1> File "TextTablePyBindGen.py", line 7, in <module>
1> module = module_parser.parse(['../TextTable/TextRow.h'])
1> File "C:\Python27\pybindgen\gccxmlparser.py", line 586, in parse
1> pygen_classifier, gccxml_options)
1> File "C:\Python27\pybindgen\gccxmlparser.py", line 683, in parse_init
1> self.declarations = parser.parse(header_files, self.gccxml_config)
1> File "C:\Python27\lib\site-packages\pygccxml\parser\__init__.py", line 50, in parse
1> answer = parser.read_files(files, compilation_mode)
1> File "C:\Python27\lib\site-packages\pygccxml\parser\project_reader.py", line 225, in read_files
1> return self.__parse_file_by_file(files)
1> File "C:\Python27\lib\site-packages\pygccxml\parser\project_reader.py", line 250, in __parse_file_by_file
1> decls = reader.read_file( header )
1> File "C:\Python27\lib\site-packages\pygccxml\parser\source_reader.py", line 197, in read_file
1> return self.read_gccxml_file( source_file )
1> File "C:\Python27\lib\site-packages\pygccxml\parser\source_reader.py", line 224, in read_gccxml_file
1> raise error
1> pygccxml.parser.source_reader.gccxml_runtime_error_t: Error occured while running GCC-XML: ../TextTable/TextRow.h:10: error: expected `)' before '*' token
1> ../TextTable/TextRow.h:36: error: expected ',' or '...' before '&' token
1> ../TextTable/TextRow.h:36: error: ISO C++ forbids declaration of 'string' with no type
1> ../TextTable/TextRow.h:45: error: ISO C++ forbids declaration of 'shared_ptr' with no type
1> ../TextTable/TextRow.h:45: error: expected ';' before '<' token
1> ../TextTable/TextRow.h:46: error: ISO C++ forbids declaration of 'weak_ptr' with no type
1> ../TextTable/TextRow.h:46: error: expected ';' before '<' token

The file TextRow.h begins as follows:

 #ifndef TEXT_ROW_H
 #define TEXT_ROW_H

 class TextRowImpl;
 class TextRowFactory;

 class TextRow {
 public:
  TextRow() {}
  TextRow(streambuf* sb, char separator, const shared_ptr<TextRowFactory>& factory);
  TextRow(const TextRow& other);
  ~TextRow();

  void swap(TextRow& other);
  TextRow& operator=(const TextRow& other);

I guess the problem is the identifiers streambuf and shared_ptr that have not been declared.
That is done in the file "StdAfx.h" that is included first in all translation units.

How do I fix that?
That problem happens before the module object is created, so module.add_include will not solve it.

Revision history for this message
Gustavo Carneiro (gjc) said :
#6

You need to write an alternative file to parse:

------- mymoduleheaders.h ------
#include <StdAfx.h>
#include "TextTable.h"
-----------------------------------------------

Then you tell pybindgen to parse mymoduleheaders.h. It's the only way.

Revision history for this message
Johan Råde (johan-rade) said :
#7

Thank you for the help. I created a header file, TextTablePyBindGen.h:

    #define BOOST_HAS_THREADS
    #include "../TextTable/StdAfx.h" // C++ std lib and Boost headers and python.h
    #include "../TextTable/TextTable.h" // my stuff
    #include "../TextTable/TextRow.h" // my stuff

Then I ran the script:

    import pybindgen
    from pybindgen import FileCodeSink
    from pybindgen.gccxmlparser import ModuleParser

    module_parser = ModuleParser('TextTablePyBindGen')
    module = module_parser.parse(['TextTablePyBindGen.h'], include_paths=['C:/Libraries/Boost/boost_1_46_1', 'C:/Python27/include'])
    module.add_include('"TextTablePyBindGen.h"')

    file = open('TextTablePyBindGen.cpp', 'w')
    pybindgen.write_preamble(FileCodeSink(file))
    module.generate(FileCodeSink(file))

The script ran without any errors and produced a cpp file.
Then I built my source code and the cpp file produced by the script to a pyd file.
When I tested the pyd file I found that it contains a module TextTablePyBindGen
but no class TextTablePyBindGen.TextTable (as defined in the header TextTable.h).

How does PyBindGen know which classes to wrap?
The header TextTablePyBindGen.h includes my code and much of the C++ standard library
and much of the Boost libraries?

Revision history for this message
Gustavo Carneiro (gjc) said :
#8

I think you need to use the option whitelist_paths of ModuleParser.parse. Sorry, I should've mentioned it but forgot.

        :param whitelist_paths: additional directories for definitions to be included
           Normally the module parser filters out API definitions that
           have been defined outside one of the header files indicated
           for parsing. The parameter whitelist_paths instructs the
           module parser to accept definitions defined in another
           header file if such header file is inside one of the
           directories listed by whitelist_paths.
        :type whitelist_paths: list of string

Revision history for this message
Johan Råde (johan-rade) said :
#9

That works. Thank you!

Another thing, the ouput from the script is not standard compliant C++.
It contains the definition

    typedef enum _PyBindGenWrapperFlags {
       PYBINDGEN_WRAPPER_FLAG_NONE = 0,
       PYBINDGEN_WRAPPER_FLAG_OBJECT_NOT_OWNED = (1<<0),
    } PyBindGenWrapperFlags;

twice. This constitutes an ODR violation. Maybe GCC will accept this
(GCC is notorious for tolerating ODR violations), but MSVC 2010 gives a compilation error.