ABI and Interfacing with Other Languages

1The calling convention

An ABI specifies what the source language does not: which registers carry arguments, who saves what, how the stack is aligned, how a returned structure is delivered. It is fixed per platform, which is what lets code from different compilers and different languages link together.

On x86-64 System V — Linux, macOS, the BSDs:

Windows x64 uses different registers entirely — rcx rdx r8 r9 — which is why a library compiled for one does not link against the other. You rarely need the details, but two facts matter in practice: a small struct may be passed in registers while a large one is passed by hidden pointer, and that threshold is part of the ABI. Changing a struct from 16 to 24 bytes can change how every function taking it is called, which is week 45's ABI break in its least obvious form.

Read it yourself with week 34's technique:

gcc -O2 -S -masm=intel example.c -o example.s

2Name mangling and extern "C"

C exports a function under its own name. C++ cannot, because it permits overloading, so it encodes the parameter types into the symbol:

/* C */    int add(int, int);   →  symbol: add
/* C++ */  int add(int, int);   →  symbol: _Z3addii

nm -D libfoo.so | c++filt        # decode mangled names

extern "C" tells a C++ compiler to use C linkage — no mangling, C calling convention — which is why the guard belongs in every public C header:

#ifdef __cplusplus
extern "C" {
#endif
    /* declarations */
#ifdef __cplusplus
}
#endif

The __cplusplus test is required because extern "C" is not valid C. With the guard, one header serves both languages; without it, a C++ program linking your library gets undefined references to mangled names that do not exist.

3Designing for foreign callers

A C API that is pleasant from C can be unusable from Python. The constraints a foreign language imposes:

The rule that follows: a good FFI surface is narrow, opaque, and explicit. Handles rather than structs, return codes rather than errno, and one free function per allocating function.

Ownership across the boundary

This is where real programs break. Python's garbage collector may free a buffer while your C code still holds the pointer; Rust's borrow checker cannot see into C at all; a Java array may be moved by the collector during a call.

• C allocates, C frees. Always export a matching free — intarray_destroy, not the caller's free(). The caller's allocator is not necessarily yours.
• Caller allocates, caller frees. Take a buffer and a size. Simplest and safest.
• Never return a pointer into memory the other language owns and expect it to stay valid.

4Worked example: one library, three callers

The C side

/* mathlib.h */
#ifndef MATHLIB_H
#define MATHLIB_H

#include <stddef.h>

#ifdef __cplusplus
extern "C" {
#endif

typedef enum {
    ML_OK = 0,
    ML_ERR_NULL     = 1,
    ML_ERR_EMPTY    = 2,
    ML_ERR_NOMEM    = 3
} MlStatus;

/* Scalars: the easiest thing to call from anywhere. */
int    ml_add(int a, int b);
double ml_mean(const double *values, size_t count, MlStatus *status);

/* A caller-supplied buffer: no ownership question at all. */
MlStatus ml_scale(const double *in, double *out, size_t count, double factor);

/* C allocates, C frees — the pair must be used together. */
double  *ml_make_range(double start, double step, size_t count);
void     ml_free(double *p);

/* An opaque handle: a foreign caller never needs the layout. */
typedef struct MlAccumulator MlAccumulator;

MlAccumulator *ml_acc_create(void);
void           ml_acc_destroy(MlAccumulator *acc);
MlStatus       ml_acc_add(MlAccumulator *acc, double value);
double         ml_acc_mean(const MlAccumulator *acc);
size_t         ml_acc_count(const MlAccumulator *acc);

/* A callback: explicit context, no hidden state. */
typedef int (*MlVisitor)(double value, void *context);
MlStatus ml_each(const double *values, size_t count,
                 MlVisitor visit, void *context);

const char *ml_status_string(MlStatus s);

#ifdef __cplusplus
}
#endif
#endif /* MATHLIB_H */

/* mathlib.c */
#include "mathlib.h"
#include <stdlib.h>

struct MlAccumulator {          /* definition stays here */
    double total;
    size_t count;
};

int ml_add(int a, int b) { return a + b; }

double ml_mean(const double *values, size_t count, MlStatus *status)
{
    if (values == NULL) { if (status) *status = ML_ERR_NULL;  return 0.0; }
    if (count == 0)     { if (status) *status = ML_ERR_EMPTY; return 0.0; }

    double total = 0.0;
    for (size_t i = 0; i < count; i++) total += values[i];
    if (status) *status = ML_OK;
    return total / (double)count;
}

MlStatus ml_scale(const double *in, double *out, size_t count, double factor)
{
    if (in == NULL || out == NULL) return ML_ERR_NULL;
    for (size_t i = 0; i < count; i++) out[i] = in[i] * factor;
    return ML_OK;
}

double *ml_make_range(double start, double step, size_t count)
{
    if (count == 0) return NULL;
    double *v = malloc(count * sizeof *v);
    if (v == NULL) return NULL;
    for (size_t i = 0; i < count; i++) v[i] = start + (double)i * step;
    return v;                      /* caller must call ml_free */
}

void ml_free(double *p) { free(p); }

MlAccumulator *ml_acc_create(void)
{
    MlAccumulator *a = calloc(1, sizeof *a);
    return a;
}

void ml_acc_destroy(MlAccumulator *a) { free(a); }

MlStatus ml_acc_add(MlAccumulator *a, double v)
{
    if (a == NULL) return ML_ERR_NULL;
    a->total += v;
    a->count++;
    return ML_OK;
}

double ml_acc_mean(const MlAccumulator *a)
{
    return (a == NULL || a->count == 0) ? 0.0 : a->total / (double)a->count;
}

size_t ml_acc_count(const MlAccumulator *a)
{
    return (a == NULL) ? 0 : a->count;
}

MlStatus ml_each(const double *values, size_t count,
                 MlVisitor visit, void *context)
{
    if (values == NULL || visit == NULL) return ML_ERR_NULL;
    for (size_t i = 0; i < count; i++) {
        if (visit(values[i], context) != 0) break;   /* non-zero stops */
    }
    return ML_OK;
}

const char *ml_status_string(MlStatus s)
{
    switch (s) {
    case ML_OK:        return "ok";
    case ML_ERR_NULL:  return "null pointer";
    case ML_ERR_EMPTY: return "empty input";
    case ML_ERR_NOMEM: return "out of memory";
    }
    return "unknown";
}

gcc -std=c17 -Wall -Wextra -O2 -fPIC -shared -o libmathlib.so mathlib.c

Caller 1 — Python with ctypes

# app.py
import ctypes, os

lib = ctypes.CDLL(os.path.abspath("libmathlib.so"))

# Declare every signature. ctypes assumes int otherwise, and a wrong
# assumption about a double is silent corruption, not an error.
lib.ml_add.argtypes = [ctypes.c_int, ctypes.c_int]
lib.ml_add.restype  = ctypes.c_int

lib.ml_mean.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.c_size_t,
                        ctypes.POINTER(ctypes.c_int)]
lib.ml_mean.restype  = ctypes.c_double

lib.ml_make_range.argtypes = [ctypes.c_double, ctypes.c_double, ctypes.c_size_t]
lib.ml_make_range.restype  = ctypes.POINTER(ctypes.c_double)
lib.ml_free.argtypes = [ctypes.POINTER(ctypes.c_double)]

lib.ml_acc_create.restype   = ctypes.c_void_p        # opaque handle
lib.ml_acc_destroy.argtypes = [ctypes.c_void_p]
lib.ml_acc_add.argtypes     = [ctypes.c_void_p, ctypes.c_double]
lib.ml_acc_mean.argtypes    = [ctypes.c_void_p]
lib.ml_acc_mean.restype     = ctypes.c_double

VISITOR = ctypes.CFUNCTYPE(ctypes.c_int, ctypes.c_double, ctypes.c_void_p)
lib.ml_each.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.c_size_t,
                        VISITOR, ctypes.c_void_p]

print("ml_add(3, 4) =", lib.ml_add(3, 4))

data = (ctypes.c_double * 5)(1.0, 2.0, 3.0, 4.0, 5.0)
status = ctypes.c_int()
print("ml_mean =", lib.ml_mean(data, 5, ctypes.byref(status)),
      "status", status.value)

# C allocated it, so C must free it.
p = lib.ml_make_range(0.0, 0.5, 6)
print("range =", [p[i] for i in range(6)])
lib.ml_free(p)

acc = lib.ml_acc_create()
for v in (10.0, 20.0, 30.0):
    lib.ml_acc_add(acc, v)
print("accumulator mean =", lib.ml_acc_mean(acc))
lib.ml_acc_destroy(acc)

# A Python function called back from C. The reference must be kept
# alive in a variable — if it is garbage collected mid-call, C jumps
# into freed memory.
total = [0.0]
def on_value(value, _ctx):
    total[0] += value
    return 0
callback = VISITOR(on_value)          # keep this name bound
lib.ml_each(data, 5, callback, None)
print("visited total =", total[0])

python3 app.py

Caller 2 — Rust

// main.rs
use std::os::raw::{c_double, c_int, c_void};

#[link(name = "mathlib")]
extern "C" {
    fn ml_add(a: c_int, b: c_int) -> c_int;
    fn ml_mean(values: *const c_double, count: usize,
               status: *mut c_int) -> c_double;
    fn ml_make_range(start: c_double, step: c_double,
                     count: usize) -> *mut c_double;
    fn ml_free(p: *mut c_double);
    fn ml_acc_create() -> *mut c_void;
    fn ml_acc_destroy(acc: *mut c_void);
    fn ml_acc_add(acc: *mut c_void, value: c_double) -> c_int;
    fn ml_acc_mean(acc: *const c_void) -> c_double;
}

fn main() {
    // Every call is unsafe: Rust cannot verify C's contracts.
    unsafe {
        println!("ml_add(3, 4) = {}", ml_add(3, 4));

        let data = [1.0_f64, 2.0, 3.0, 4.0, 5.0];
        let mut status: c_int = 0;
        let mean = ml_mean(data.as_ptr(), data.len(), &mut status);
        println!("ml_mean = {mean} status {status}");

        let p = ml_make_range(0.0, 0.5, 6);
        let slice = std::slice::from_raw_parts(p, 6);
        println!("range = {slice:?}");
        ml_free(p);                       // C's allocator, not Rust's

        let acc = ml_acc_create();
        for v in [10.0, 20.0, 30.0] {
            ml_acc_add(acc, v);
        }
        println!("accumulator mean = {}", ml_acc_mean(acc));
        ml_acc_destroy(acc);
    }
}

rustc -L . main.rs -o rustapp
LD_LIBRARY_PATH=. ./rustapp

Caller 3 — C++

// app.cpp
#include "mathlib.h"        // the extern "C" guard does the work
#include <iostream>
#include <memory>
#include <vector>

int main()
{
    std::cout << "ml_add(3, 4) = " << ml_add(3, 4) << '\n';

    std::vector<double> data{ 1.0, 2.0, 3.0, 4.0, 5.0 };
    MlStatus status;
    double mean = ml_mean(data.data(), data.size(), &status);
    std::cout << "ml_mean = " << mean
              << " (" << ml_status_string(status) << ")\n";

    // Wrap the C handle so the destructor cannot be forgotten.
    auto acc = std::unique_ptr<MlAccumulator, decltype(&ml_acc_destroy)>(
                   ml_acc_create(), &ml_acc_destroy);
    for (double v : { 10.0, 20.0, 30.0 }) {
        ml_acc_add(acc.get(), v);
    }
    std::cout << "accumulator mean = " << ml_acc_mean(acc.get()) << '\n';

    return 0;
}

g++ -std=c++17 app.cpp -L. -lmathlib -o cppapp
LD_LIBRARY_PATH=. ./cppapp

Prove the mangling

nm -D libmathlib.so | grep ' T ' | head

0000000000001130 T ml_acc_add
0000000000001100 T ml_acc_create
…
0000000000001110 T ml_add

Plain names. Now compile the same source as C++ without the guard and look again:

g++ -fPIC -shared -x c++ mathlib.c -o libmangled.so 2>/dev/null
nm -D libmangled.so | grep ' T ' | head -3
nm -D libmangled.so | grep ' T ' | head -3 | c++filt

_Z6ml_addii rather than ml_add. No FFI in any language would find it, and that single difference is what the four lines of extern "C" prevent.

Break the ownership rule on purpose

In the Python script, replace lib.ml_free(p) with nothing and run under a leak checker; then try freeing C's pointer from Python's side. On Linux with one shared libc it may appear to work; on Windows, where the library and the interpreter can use different runtimes, it corrupts the heap. The exported ml_free exists precisely so the caller never has to know which allocator produced the block.

Then delete the callback = VISITOR(on_value) binding and pass VISITOR(on_value) inline. Python may collect the temporary while C still holds the pointer, and the call jumps into freed memory. Keeping the reference alive for the duration of the call is the FFI equivalent of week 13's dangling pointer.

5Inline assembly and intrinsics

Occasionally there is no C expression for what you need — a specific instruction, a control register, an atomic primitive the compiler does not expose.

/* GCC extended asm */
static inline uint64_t read_timestamp(void)
{
    uint32_t low, high;
    __asm__ volatile ("rdtsc" : "=a"(low), "=d"(high));
    return ((uint64_t)high << 32) | low;
}

The syntax is: instructions, then outputs, then inputs, then clobbers. volatile prevents the compiler moving or eliding it.

Prefer an intrinsic. Compiler intrinsics expose the same instructions as ordinary functions, so the optimizer can still schedule and inline around them:

#include <immintrin.h>
uint64_t t = __rdtsc();                      /* same instruction */

int bits = __builtin_popcount(x);            /* week 25, in one instruction */
if (__builtin_add_overflow(a, b, &sum)) { }  /* week 36's check, correctly */

That last one is worth noting: __builtin_add_overflow performs the addition and reports overflow without ever invoking undefined behavior — the problem week 36 spent a section on. C23 standardizes equivalents in <stdckdint.h>.

Inline assembly is not portable across architectures or even compilers, it blocks optimization, and it is easy to get the constraints wrong. Reach for it only when no intrinsic exists.

6Common mistakes

8Where this leads

Week 47 returns to the socket server of week 40 and rewrites it as a single-threaded event loop with epoll, then measures both under load. That is the last piece of the systems-programming material before the embedded block begins.