Python C-api Object Allocation

I want to use the new and delete operators for creating and destroying my objects. The problem is python seems to break it into several stages. tp_new, tp_init and tp_alloc for cre

Solution 1:

The documentation for these is at http://docs.python.org/3.0/c-api/typeobj.html and http://docs.python.org/3.0/extending/newtypes.html describes how to make your own type.

tp_alloc does the low-level memory allocation for the instance. This is equivalent to malloc(), plus initialize the refcnt to 1. Python has it's own allocator, PyType_GenericAlloc, but a type can implement a specialized allocator.

tp_new is the same as Python's __new__. It's usually used for immutable objects where the data is stored in the instance itself, as compared to a pointer to data. For example, strings and tuples store their data in the instance, instead of using a char * or a PyTuple *.

For this case, tp_new figures out how much memory is needed, based on the input parameters, and calls tp_alloc to get the memory, then initializes the essential fields. tp_new does not need to call tp_alloc. It can for example return a cached object.

tp_init is the same as Python's __init__. Most of your initialization should be in this function.

The distinction between __new__ and __init__ is called two-stage initialization, or two-phase initialization.

You say "c++ just has new" but that's not correct. tp_alloc corresponds a custom arena allocator in C++, __new__ corresponds to a custom type allocator (a factory function), and __init__ is more like the constructor. That last link discusses more about the parallels between C++ and Python style.

Also read http://www.python.org/download/releases/2.2/descrintro/ for details about how __new__ and __init__ interact.

You write that you want to "create the object directly in c++". That's rather difficult because at the least you'll have to convert any Python exceptions that occurred during object instantiation into a C++ exception. You might try looking at Boost::Python for some help with this task. Or you can use a two-phase initialization. ;)

Solution 2:

I don't know the python APIs at all, but if python splits up allocation and initialization, you should be able to use placement new.

e.g.:

// tp_allocvoid *buffer = newchar[sizeof(MyExtensionObject)];
 // tp_init or tp_new (not sure what the distinction is there)new (buffer) MyExtensionObject(args);
 returnstatic_cast<MyExtensionObject*>(buffer);

 ...
 // tp_del
 myExtensionObject->~MyExtensionObject(); // call dtor// tp_dealloc (or tp_free? again I don't know the python apis)delete [] (static_cast<char*>(static_cast<void*>(myExtensionObject)));