Cython - Advanced topics

In this article I will continue the introduction article on Cython and show some usages of calling Python code from C code. That's because on Reddit someone asked for a more detailed tutorial on other advanced features.

We will create a simple example which embeds Python code into C code and creates an executable application which can be run on your system and includes the Python interpreter so there is no need to call python.

As a prerequisite you should have Python 3 and a C compiler installed because we will compile C code to native executables as mentioned previously. Without a C compiler you will not be able to try out the examples in this article.

I have written an article on my Mac environment so perhaps the code examples will not work as you would expect them to. If you encounter a problem, on Windows for example, just let me know and I'll try to add an update to the article to give you the opportunity to try the code.

Embedding and compiling Python code

First we have to take a look at how the Python initialization API and the link load model work when embedding Python code into C or C++.

To embed Python code we will simply use this skeleton:

#include <Python.h>
// include your module's header here

int main (int argc, char **argv) {

   // initialization
   Py_SetProgramName (argv [0]);
   Py_Initialize ();
   //PyInit_yourfilename(); // Initialization for Python 3

   // logic goes here

   // last two lines of the main function
   Py_Finalize();
   return 0;
}

Make sure that the #include <Python.h> is the first include in your C/C++ files because Python itself uses many header files which turn settings on or off. After this there is a very simple main function which has all the logic you want to add. Naturally you do not have to write all the code inside just this function—you can create others to follow best coding practices.

And as you can see, there are three lines which do initialization. The first two speak for themselves and are the same every time but the third one has to be changed for every module you want to use. In the sample above it is commented out and has a "funny" name but later I will show you an example where it will make sense and you'll understand how this line should work.

There are two ways in which you can add Python features to C code. One way is to share Python objects. This is a Pythonic approach and I suggest that you use this one when your codebase is mostly Python code. In this case you will have a native module which calls native code. The other way is to fully embed Python and Python code into your native application. This means in this case we already have the Python runtime environment—in the other solution we had to run the Python interpreter which called our Cython code (this is what we did in the previous article).

Now that we know how to link Python code to run, let's take a deeper look at the following code samples:

cdef void cython_function():
   print('inside a Cython function')

cdef public void cython_function():
   print('inside a Cython function')

The only difference is the public keyword. But this makes a very significant difference. If we compile both with cythonize we get the following result:

As you can see, the code containing the public keyword gets beside its .c file a .h file too. This header file contains a line which declares a function initcythonfile_public(void) if the Python version is smaller than 2. For Python 3 there is nothing more to do but for Python 2 you have to call this initializer method in the boilerplate code mentioned at the beginning of this section before you do any more calls.

Now let's create our C file which will call the previously defined function:

#include <Python.h>
#include "cythonfile_public.h"

int main (int argc, char ** argv) {

   Py_SetProgramName (argv [0]);
   Py_Initialize ();
   PyInit_cythonfile_public();

   // logic goes here
   cython_function ();

   Py_Finalize();
   return 0;
}

Compilation of the example goes with the following steps:

cython -3 cythonfile_public.pyx
gcc -g -fpic -c cythonfile_public.c -o cythonfile_public.o `python3.6-config --includes`
gcc -g -fpic -c cythontest.c -o cythontest.o `python3.6-config --includes`
gcc -g -o cythontest cythonfile_public.o cythontest.o `python3.6-config --libs` -L /Library/Frameworks/Python.framework/Versions/3.6/lib

The first command compiles the cythonfile_public.pyx Cython file to a .h and a .c file. The -3 flag tells the Cython compiler that the Python code was written with Python 3 syntax and a Python 3 based C file will be created.

The second command compiles this C file, the third command compiles the cythontest.c file and the fourth command links the two compiled .o files to a runnable called cythontest.

The executed sub-commands like python3.6-config --includes provide the location of the libraries of the Python installation which should be included in the compilation process. The name python3.6-config includes the version number of my Python installation because I have many and my default is not this one. You can verify if python-config is sufficient for your computer by executing this command:

GHajba$ python3.6-config --includes
-I/Library/Frameworks/Python.framework/Versions/3.6/include/python3.6m -I/Library/Frameworks/Python.framework/Versions/3.6/include/python3.6m

You can see the version number in the output. If it is greater than 3 then you are good to go. If it is something like this:

GHajba$ python-config --includes
-I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7

then you have to adjust your configuration to point to a Python 3 installation for these examples.

The compilation-command-block above can be included into a Makefile too if you do not want to type all this every time.

We can execute the sample application by running the runnable created previously. It looks like this on my computer:

GHajba$ ./cythontest
inside a Cython function

As you can see, the linking of the Cython code and the C code works smoothly.

Conclusion

In this article we have taken the next step in using Python code from C code with the Cython module. We have seen how we can compile parts together to have a native executable based on a C code which calls a function in a Cython module.

Compare the best development tools with DiscoverSDK.