CPython Build System: How configure and Makefile Are Generated¶

Overview¶

CPython uses the GNU Autotools build system to support compilation across Linux, macOS, Windows (via WSL/Cygwin), FreeBSD, and more. The core idea is:

Write once in portable macros → generate platform-specific shell scripts → detect system capabilities → produce a tailored Makefile.

The pipeline looks like this:

flowchart TD
    A["configure.ac\naclocal.m4"] -->|autoconf / autoheader| B["configure\npyconfig.h.in"]
    B -->|./configure run by developer| C["Makefile.pre\npyconfig.h"]
    D["Modules/Setup"] --> E
    C -->|make makesetup| E["Makefile"]

1. Source Files (What Developers Maintain)¶

configure.ac¶

The heart of the build system. Written in M4 macro language mixed with shell script. CPython's configure.ac is ~8,000 lines long and lives at the root of the repo.

Key responsibilities:

• Declares the project name and required Autoconf version
• Detects the C compiler, linker, and their flags
• Probes for system headers, libraries, and functions
• Handles cross-compilation settings
• Generates #define values for pyconfig.h

Example snippet from configure.ac:

AC_INIT([python], [3.13])
AC_PREREQ([2.71])

# Check for C compiler
AC_PROG_CC

# Check if we have a working pthread
AC_CHECK_LIB([pthread], [pthread_create])

# Check for specific headers
AC_CHECK_HEADERS([fcntl.h sys/file.h sys/socket.h])

# Custom CPython macro (defined in aclocal.m4)
PY_CHECK_FUNC([clock_gettime], [time.h])

aclocal.m4¶

Contains custom M4 macros specific to CPython that extend what stock Autoconf provides. These macros are prefixed with PY_ by convention.

Examples of CPython-specific macros:

Makefile.pre.in¶

A template for the final Makefile. Autoconf substitutes @VARIABLE@ placeholders with values detected at configure time.

# In Makefile.pre.in (template)
CC=         @CC@
CXX=        @CXX@
OPT=        @OPT@
LDFLAGS=    @LDFLAGS@
prefix=     @prefix@
LIBDIR=     @libdir@

# Becomes in Makefile (after ./configure)
CC=         gcc
CXX=        g++
OPT=        -O2 -g
LDFLAGS=    -L/usr/local/lib
prefix=     /usr/local
LIBDIR=     /usr/local/lib

pyconfig.h.in¶

A template for the C header pyconfig.h. Autoconf fills in #define / #undef entries based on what was detected on the system.

/* In pyconfig.h.in */
#undef HAVE_FCNTL_H
#undef HAVE_PTHREAD_H
#undef SIZEOF_LONG

/* Becomes in pyconfig.h (after ./configure on a Linux x86-64) */
#define HAVE_FCNTL_H 1
#define HAVE_PTHREAD_H 1
#define SIZEOF_LONG 8

2. Generating configure (What Release Managers Do)¶

The configure shell script is not stored in version control in CPython's main repo — it is generated from configure.ac by running:

autoreconf -ivf

This invokes:

What autoconf does internally¶

autoconf expands all AC_* and PY_* macros from configure.ac into a portable POSIX shell script (~20,000+ lines) that:

• Accepts -prefix, -enable-*, -with-* flags from the user
• Runs compiler feature tests by compiling small C snippets
• Writes detected values into config.status

3. Running ./configure (What Developers Do)¶

When you run ./configure, the generated shell script executes on your specific machine and does the following:

Phase 1 — Argument parsing¶

./configure \
  --prefix=/usr/local \
  --enable-optimizations \
  --with-lto \
  --enable-shared \
  --with-openssl=/usr/local/opt/openssl

Common flags:

Phase 2 — Feature detection¶

configure compiles hundreds of small C programs to test capabilities. For example:

# configure tests if clock_gettime is available
cat > conftest.c << 'EOF'
#include <time.h>
int main() { struct timespec ts; clock_gettime(CLOCK_REALTIME, &ts); return 0; }
EOF
gcc -o conftest conftest.c

If the test compiles and links, configure sets HAVE_CLOCK_GETTIME=1 in pyconfig.h.

Phase 3 — Output generation¶

configure produces three key outputs:

1. Makefile.pre — intermediate Makefile (from Makefile.pre.in)
2. pyconfig.h — C header with all detected #defines
3. config.status — a shell script that can regenerate outputs without re-running all tests

4. From Makefile.pre to Makefile¶

CPython has a second stage of Makefile generation that most projects don't need. After configure produces Makefile.pre, running make triggers:

make -f Makefile.pre Makefile

This calls a Python script called makesetup:

Makefile.pre + Modules/Setup  →  makesetup  →  Makefile

Modules/Setup¶

This file lists which C extension modules should be built statically (compiled into python itself) vs dynamically (compiled as .so files):

# Modules/Setup (simplified)

# Format: module_name  source_files  compiler_flags  libraries

# Built-in (static) modules
posix   posixmodule.c
errno   errnomodule.c
_sre    _sre/sre.c

# Dynamic modules (compiled as .so)
*shared*
_csv    _csv.c
_json   _json.c   # links nothing extra
_ssl    _ssl.c  -I$(OPENSSL_INCLUDES) -L$(OPENSSL_LIBDIR) -lssl -lcrypto

What makesetup does¶

makesetup is a Python script (Modules/makesetup) that:

1. Reads Modules/Setup line by line
2. Generates make rules for each module
3. Appends those rules to Makefile.pre to produce the final Makefile

This means the final Makefile has explicit targets like:

Modules/_ssl.cpython-313-x86_64-linux-gnu.so: Modules/_ssl.c
    $(CC) $(CCSHARED) $(CFLAGS) -I/usr/local/opt/openssl/include \
      -c Modules/_ssl.c -o Modules/_ssl.o
    $(LDSHARED) Modules/_ssl.o -L/usr/local/opt/openssl/lib \
      -lssl -lcrypto -o $@

5. Cross-Compilation Support¶

For building CPython on one platform to run on another (e.g., building for ARM on an x86 machine):

./configure \
  --build=x86_64-linux-gnu \   # machine running the compiler
  --host=aarch64-linux-gnu \   # target machine
  --prefix=/opt/python-arm

Autoconf uses the --build / --host / --target triplet convention. CPython's configure.ac includes guards like:

if test "$cross_compiling" = yes; then
  # Cannot run test programs, use cached or conservative values
  AC_MSG_WARN([Cross compiling: assuming SIZEOF_LONG=8])
  ac_cv_sizeof_long=8
fi

A config.site file can pre-seed these cached values:

# config.site for ARM cross-compilation
ac_cv_file__dev_ptmx=yes
ac_cv_file__dev_ptc=no
ac_cv_sizeof_long=8
ac_cv_sizeof_size_t=8

6. Tracing the Full Flow: End to End¶

CPython source checkout
│
├── configure.ac          ← M4 macro script
├── aclocal.m4            ← Custom PY_* macros
├── Makefile.pre.in       ← Makefile template
├── pyconfig.h.in         ← C header template
│
│   [autoconf/autoheader — run by release manager]
│        ↓
├── configure             ← generated POSIX shell script (~20k lines)
│
│   [./configure — run by developer]
│        ↓
├── Makefile.pre          ← partially filled Makefile
├── pyconfig.h            ← platform-specific C header
├── config.status         ← regeneration script
│
│   [make -f Makefile.pre Makefile]
│   [calls Modules/makesetup]
│        ↓
├── Makefile              ← final build script with module rules
│
│   [make]
│        ↓
└── python                ← compiled interpreter

7. Useful Developer Commands¶

# Regenerate configure from configure.ac (needs autoconf >= 2.71)
autoreconf -ivf

# See all available ./configure options
./configure --help

# Configure with common developer flags
./configure --with-pydebug --without-pymalloc CFLAGS="-O0 -g"

# Regenerate Makefile without re-running ./configure
./config.status

# Regenerate only pyconfig.h
./config.status pyconfig.h

# See what configure detected (all substituted variables)
./config.status --version
grep "^#define" pyconfig.h | head -40

# Verbose make (see exact compiler invocations)
make V=1

# Build only a specific extension module
make Modules/_ssl.cpython-313-x86_64-linux-gnu.so

8. Key Files Reference¶