Building a High-Performance C Extension for Python¶

Python is beloved for its readability and ease of use, but it isn't always the fastest tool for heavy numerical lifting. When you hit a performance bottleneck, you can drop down into C to handle the "heavy lifting" while keeping your high-level logic in Python.

This guide walks you through creating a C extension from scratch.

1. The Architecture of a C Extension¶

Before writing code, it helps to understand how Python (written in C) interacts with your custom C code. Every extension requires a specific boilerplate structure to "handshake" with the Python interpreter.

The 4 Pillars of a C Extension:¶

1. The Header: Including <Python.h> to access the Python API.
2. The Logic: Your standard C functions.
3. The Mapping: A method table that maps C functions to Python names.
4. The Initialization: A function that registers the module when you call import.

2. Writing the C Source (power.c)¶

Let’s create a module called power_tools that calculates the square of a number. This is a simple example, but the pattern scales to complex algorithms.

#include <Python.h>

/**
 * 1. The Method Function
 * This is where the work happens. 
 * 'self' points to the module object.
 * 'args' contains the arguments passed from Python.
 */
static PyObject* method_square(PyObject* self, PyObject* args) {
    double value;

    // Parse the Python arguments into C types
    // "d" tells Python to expect a double (float)
    if (!PyArg_ParseTuple(args, "d", &value)) {
        return NULL; // Raises a TypeError in Python
    }

    double result = value * value;

    // Convert the C double back into a Python Float object
    return PyFloat_FromDouble(result);
}

/**
 * 2. The Method Table
 * Tells Python which C functions are available.
 */
static PyMethodDef PowerMethods[] = {
    {"square", method_square, METH_VARARGS, "Returns the square of a number."},
    {NULL, NULL, 0, NULL} // Sentinel
};

/**
 * 3. The Module Definition
 */
static struct PyModuleDef powertools_module = {
    PyModuleDef_HEAD_INIT,
    "power_tools",       // Module name
    "A module for fast power calculations.", 
    -1, 
    PowerMethods
};

/**
 * 4. The Initialization Function
 * Must be named PyInit_<modulename>
 */
PyMODINIT_FUNC PyInit_power_tools(void) {
    return PyModule_Create(&powertools_module);
}

3. The Build Script (setup.py)¶

Compiling C for Python manually is tedious. Python’s setuptools handles the compiler flags, include paths, and linking for you.

from setuptools import setup, Extension

# Define the extension module
ext_module = Extension(
    "power_tools", 
    sources=["power.c"]
)

setup(
    name="PowerTools",
    version="1.0",
    description="C Extension Tutorial",
    ext_modules=[ext_module],
)

4. Compiling and Using the Module¶

Step A: Compile¶

Run the following in your terminal. The --inplace flag puts the compiled .so (Linux/Mac) or .pyd (Windows) file in your current directory.

python setup.py build_ext --inplace

Step B: Test in Python¶

Now, you can import it like any other library:

import power_tools

val = 5.0
print(f"The square of {val} is {power_tools.square(val)}")