Call Python function From C!
August 2022
2004 Words, 12 Minutes
CPython is one of the many python runtimes. It contains both a runtime and a shared language specification that all the python runtimes use. Python’s official implementation is in C, which is referred as CPython.
This means that we can make use of the Python features in C and vice-versa using the CPython API . The scope of this blog will be limited to calling python functions from C. For detailed information please refer the docs
Let’s start with a python function that we want to call from C. We have the following area function defined in Rectangle.py which calculates the area of a rectangle.
1
2
def area(a, b):
return a*b
Now our goal is to execute this python function from C by making use of the CPython API. First I will add the C code below, which calls the python function and then I will explain each and every line of the code to make it clear to the reader. The code itself is documented as well.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
#include<Python.h>
double call_func(PyObject *func, double x, double y)
{
PyObject *args;
PyObject *kwargs;
PyObject *result = 0;
double retval;
//https://docs.python.org/3/c-api/call.html#call-support-api
// Verify that func is a callable
if (!PyCallable_Check(func))
{
fprintf(stderr, "call_func: expected a callable\n");
goto fail;
}
// https://docs.python.org/3/c-api/arg.html#building-values
// Build args tuple which will be passed to python func
args = Py_BuildValue("(dd)", x, y);
kwargs = NULL;
// https://docs.python.org/3/c-api/call.html#c.PyObject_Call
// Call the python function
result = PyObject_Call(func, args, kwargs);
// https://docs.python.org/3/c-api/refcounting.html#c.Py_DecRef
// Decrease the referece since Py_BuildValue returns a new ref.
Py_DECREF(args);
// https://docs.python.org/3/c-api/refcounting.html#c.Py_XDECREF
// Decrease the reference using XDECREF because kwargs is NULL
Py_XDECREF(kwargs);
// https://docs.python.org/3/c-api/exceptions.html#c.PyErr_Occurred
// Check if python function was executed without any error.
if (PyErr_Occurred())
{
PyErr_Print();
goto fail;
}
// https://docs.python.org/3/c-api/float.html#c.PyFloat_Check
// Verify the result is a float object
if (!PyFloat_Check(result))
{
fprintf(stderr, "call_func: callable didn't return a float\n");
goto fail;
}
// https://docs.python.org/3/c-api/float.html#c.PyFloat_AsDouble
// Convert PyFloat to C double
retval = PyFloat_AsDouble(result);
Py_DECREF(result);
return retval;
fail:
Py_XDECREF(result);
abort();
}
/* Load a symbol from a module */
PyObject *import_function(const char *modname, const char *symbol)
{
PyObject *u_name, *module;
// https://docs.python.org/3/c-api/unicode.html#c.PyUnicode_FromString
// Convert Unicode Object from UTF-8 encoded char buffer.
u_name = PyUnicode_FromString(modname);
if(u_name == NULL){
printf("Couldnot convert string to unicode\n");
return NULL;
}
// https://docs.python.org/3/c-api/import.html#c.PyImport_Import
// Import the module, PyImport_Import performs absolute import
module = PyImport_Import(u_name);
if(module == NULL){
printf("Couldnot Import Module\n");
return NULL;
}
// https://docs.python.org/3/c-api/object.html#c.PyObject_GetAttrString
// Get the function from module
// This is the equivalent of the Python expression o.attr_name.
return PyObject_GetAttrString(module, symbol);
}
int main()
{
PyObject *area_func;
double x,y;
// https://docs.python.org/3/c-api/init.html#c.Py_Initialize
// Initialize the Python interpreter and
// since version 3.7 it also create the GIL explicitly by calling PyEval_InitThreads()
// so you don’t have to call PyEval_InitThreads() yourself anymore
Py_Initialize();
// Get a reference to the Area.area function
area_func = import_function("Rectangle", "area");
if(area_func == NULL){
return -1;
}
// Call it using our call_func() code
for (x = 0.0,y=10.0; x <= 10.0 && y>=0 ; x += 0.5, y-=0.5)
{
printf("x : % 0.2f y : % 0.2f Area : % 0.2f\n", x, y, call_func(area_func, x, y));
}
Py_DECREF(area_func);
// Undo all initializations made by Py_Initialize() and subsequent
Py_Finalize();
return 0;
}
On line 1 we include the Python.h header which is the entry point of Python C API.
On line 102 inside the main function we Initialize the python interpreter. Since we are executing the python code from C we need to initialize the interpreter where we can execute the python byte code. This line will Initialize the python Interpreter and will create the GIL explicitly by calling PyEval_InitThreads and lock it.
Any python operation that we perform on the interpreter should be done only when the GIL is locked. Python interpreter is not fully thread safe hence we need to take care of GIL here as we are performing python operations from C. More information about the GIL can be found here.
The convention that we follow while performing a python operation is as follows:
- Lock the GIL
- Do Python
- Unlock the GIL
In the code above the GIL is locked when we initialize the interpreter, so we don’t lock it explicitly anywhere in the whole program because the above program is single threaded.
On line no. 105 we import the area function from Rectangle.py. Inside the import_function on line no 64 takes module name and function name as arguments. It converts module name to PyObject using PyUnicode_FromString on line 70 which is passed as an argument to PyImport_Import on line 78. After we import the module we get the required function on line no 89 using PyObject_GetAttrString, this function is an equivalent of Python expression o.attr_name, where o is the object and attr_name is the attribute that we want to get.
After we import the area function its time to call the function from C. On line no 111-114 we have a for loop that has two variables x and y which we pass as an argument to call_func, we also pass the area_func which is our area function in Rectangle.py as an argument to call_func.
call_func is defined on line no. 3. In this function we first check whether the func that we have passed is callable or not. PyObject_Call on line 25 calls the python function with args and kwargs, in order to make this function call first we prepare these arguments. On line 20 we build a tuple with x and y which are our positional arguments which we pass as args, kwargs is NULL because we don’t have any keyword arguments in our area function. This is where we call our area function and get its output.
Before we further go thorugh the code, We need to know about Python’s Memory management which you can read here. This is a pre-requisite that needs to be understood before we go through the other section of call_func.
Py_BuildValue on line no 20 returns a new reference to a PyObject, everytime a new reference to a PyObject is created in memory, it’s the responsibility of the programmer to decrease its reference count. If the object’s reference count is not decreased then the Python Garbage Collector won’t remove that object from the memory and we will end up having a memory leak.
After the args is used we decrease it’s reference count on line no 29 using Py_DECREF. We use Py_XDECREF to decrease the reference count of kwargs. In this case kwargs is NULL and we need not decrease any refernce count here, but using Py_XDECREF we demonstrate that it is possible to decrease the reference of those objects whose value is NULL. Whenever we are not sure of the value of the PyObject, we should use Py_XDECREF. More information of these functions can be found here
On line 37 we make sure that PyObject_Call ran successfully without any errors, if there is any runtime error on python side, then we catch it and print it on line 39 using PyErr_Print.
On line 45 we check the result’s type and make sure that it is a float object. On line 53 we convert the PyFloat object to C double using PyFloat_AsDouble and then we decrease the reference count of the result and we return the retval on line 56.
After completion of the loop on line 111-114 we decrease the reference count of area_func and the call Py_Finalize, this will undo all the initializations made by Py_Initialize.
How to run the above code
Create the two files add the respective C and Python code and save them in the same dir. In my case both the files are in cpython dir. Then follow the given commands to execute the code successfully.
cd cpython
export PYTHONPATH=.
cc -g ctopy.c -I/usr/include/python3.10 -L /usr/lib/python3.10/config-3.10-x86_64-linux-gnu -lpython3.10
The path to python interpreter might be different in your system, you would have to change that according to your requirements.
Output
./a.out
x : 0.00 y : 10.00 Area : 0.00
x : 0.50 y : 9.50 Area : 4.75
x : 1.00 y : 9.00 Area : 9.00
x : 1.50 y : 8.50 Area : 12.75
x : 2.00 y : 8.00 Area : 16.00
x : 2.50 y : 7.50 Area : 18.75
x : 3.00 y : 7.00 Area : 21.00
x : 3.50 y : 6.50 Area : 22.75
x : 4.00 y : 6.00 Area : 24.00
x : 4.50 y : 5.50 Area : 24.75
x : 5.00 y : 5.00 Area : 25.00
x : 5.50 y : 4.50 Area : 24.75
x : 6.00 y : 4.00 Area : 24.00
x : 6.50 y : 3.50 Area : 22.75
x : 7.00 y : 3.00 Area : 21.00
x : 7.50 y : 2.50 Area : 18.75
x : 8.00 y : 2.00 Area : 16.00
x : 8.50 y : 1.50 Area : 12.75
x : 9.00 y : 1.00 Area : 9.00
x : 9.50 y : 0.50 Area : 4.75
x : 10.00 y : 0.00 Area : 0.00
References: