Event-Driven and Scalable I/O

1Why blocking does not scale

A blocking read suspends the calling thread until data arrives. With one thread per connection that is fine — the thread has nothing else to do. The costs appear at scale:

The memory is only half of it. Ten thousand runnable threads make the scheduler's work proportional to the connection count, and every context switch costs cache locality — week 34's subject, at the operating system's scale.

An event loop turns the problem inside out. Instead of waiting for one descriptor, ask the kernel which of these are ready, then service exactly those, then ask again.

2Non-blocking descriptors

#include <fcntl.h>

static bool set_nonblocking(int fd)
{
    int flags = fcntl(fd, F_GETFL, 0);
    if (flags < 0) return false;
    return fcntl(fd, F_SETFL, flags | O_NONBLOCK) == 0;
}

On a non-blocking descriptor, an operation that would have waited returns −1 with errno set to EAGAIN (or the identical EWOULDBLOCK). That is not an error — it means "nothing right now, ask again later".

ssize_t n = recv(fd, buf, sizeof buf, 0);
if (n > 0)                                  { /* got data */ }
else if (n == 0)                            { /* peer closed */ }
else if (errno == EAGAIN || errno == EWOULDBLOCK) { /* not ready */ }
else if (errno == EINTR)                    { /* retry */ }
else                                        { /* real error */ }

Those five cases are the whole of non-blocking I/O, and treating EAGAIN as a failure is the most common beginner error.

3The three multiplexers

The complexity row is the reason epoll exists. With select or poll, every call passes the entire descriptor list to the kernel, which examines all of it — so a server with 10 000 idle connections and one active does 10 000 units of work per event. epoll registers the set once and returns only what is ready.

The equivalents elsewhere are kqueue on the BSDs and macOS, and IOCP on Windows. Portable code uses libevent, libuv, or libev, which wrap all of them behind one interface. Writing directly to epoll once is still worth doing, because it makes the libraries comprehensible.

#include <sys/epoll.h>

int ep = epoll_create1(0);

struct epoll_event ev = { .events = EPOLLIN, .data.fd = listener };
epoll_ctl(ep, EPOLL_CTL_ADD, listener, &ev);

struct epoll_event ready[64];
int n = epoll_wait(ep, ready, 64, timeout_ms);
for (int i = 0; i < n; i++) {
    handle(ready[i].data.fd, ready[i].events);
}

Level-triggered and edge-triggered

Edge-triggered mode is faster because it reports each event once, and it is unforgiving: read half the available data and you will not be told again. Start level-triggered.

4Structuring the loop

Blocking code keeps its state on the stack — the local variables of the function handling the connection. An event loop cannot: it returns to the top after each event, so every connection needs an explicit state object.

typedef enum { ST_READING_HEADER, ST_READING_BODY, ST_WRITING } State;

typedef struct {
    int     fd;
    State   state;
    char    in[4096];
    size_t  in_used;
    char    out[4096];
    size_t  out_used, out_sent;
    time_t  last_active;
} Connection;

This is the real cost of the model: what was a local variable becomes a field, and what was a sequence of statements becomes a state machine. In exchange you get a thousand connections in one thread with no locking anywhere.

Two rules that are easy to get wrong:

Never block inside the loop. One blocking call — a synchronous DNS lookup, a disk read, a slow malloc — stalls every connection, not one.

Only ask for writability when you have something to write. A descriptor is almost always writable, so leaving EPOLLOUT registered makes epoll_wait return immediately, forever. Add it when your output buffer is non-empty and remove it when the buffer drains.

5Timers and signals

Both need to be part of the loop rather than outside it.

The simplest timer is the epoll_wait timeout: compute the time until the nearest deadline, pass it, and sweep for expiries when the call returns. Linux also offers timerfd_create, which makes a timer into a descriptor the loop can wait on like any other.

Signals are harder. Week 39 established that a handler may do almost nothing — so the classic solution makes the signal into an event:

/* the self-pipe trick */
static int wake[2];                      /* pipe(wake) at startup */

static void on_signal(int sig)
{
    (void)sig;
    char b = 1;
    ssize_t r = write(wake[1], &b, 1);   /* write IS async-signal-safe */
    (void)r;
}
/* register wake[0] with epoll; the loop now sees signals as events */

Linux provides signalfd, which does the same thing without the pipe. Either way the signal is handled in the loop's ordinary flow, where the full standard library is available.

6Worked example: the week 40 server, rewritten

#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdbool.h>
#include <errno.h>
#include <time.h>
#include <unistd.h>
#include <fcntl.h>
#include <signal.h>
#include <sys/epoll.h>
#include <sys/socket.h>
#include <netdb.h>

#define MAX_EVENTS   64
#define BUF_SIZE   4096
#define IDLE_LIMIT   30      /* seconds */

typedef struct {
    int     fd;
    char    in[BUF_SIZE];
    size_t  in_used;
    char    out[BUF_SIZE];
    size_t  out_used;
    size_t  out_sent;
    long    lines;
    time_t  last_active;
    bool    closing;
} Connection;

static Connection *conns[65536];
static int  epfd     = -1;
static int  wake[2]  = { -1, -1 };
static long live     = 0;
static long peak     = 0;
static long total    = 0;

/* ---------- signal handling via the self-pipe trick ---------- */

static void on_signal(int sig)
{
    (void)sig;
    char b = 1;
    ssize_t r = write(wake[1], &b, 1);    /* the only safe call here */
    (void)r;
}

static bool set_nonblocking(int fd)
{
    int flags = fcntl(fd, F_GETFL, 0);
    if (flags < 0) return false;
    return fcntl(fd, F_SETFL, flags | O_NONBLOCK) == 0;
}

/* ---------- connection lifecycle ---------- */

static void conn_close(Connection *c)
{
    epoll_ctl(epfd, EPOLL_CTL_DEL, c->fd, NULL);
    close(c->fd);
    conns[c->fd] = NULL;
    free(c);
    live--;
}

/* Ask for writability only while there is something to write. */
static void conn_update_interest(Connection *c)
{
    struct epoll_event ev = { .data.fd = c->fd, .events = EPOLLIN };
    if (c->out_used > c->out_sent) {
        ev.events |= EPOLLOUT;
    }
    epoll_ctl(epfd, EPOLL_CTL_MOD, c->fd, &ev);
}

static void conn_queue(Connection *c, const char *data, size_t n)
{
    if (c->out_used + n > BUF_SIZE) {      /* client not draining */
        c->closing = true;
        return;
    }
    memcpy(c->out + c->out_used, data, n);
    c->out_used += n;
}

/* Framing, exactly as in week 40: a stream has no message boundaries. */
static void conn_process_input(Connection *c)
{
    char *start = c->in;
    char *nl;
    while ((nl = memchr(start, '\n', (size_t)(c->in + c->in_used - start)))) {
        size_t len = (size_t)(nl - start);
        c->lines++;

        if (len == 4 && memcmp(start, "quit", 4) == 0) {
            conn_queue(c, "bye\n", 4);
            c->closing = true;
            break;
        }

        char reply[BUF_SIZE];
        int n = snprintf(reply, sizeof reply, "%ld: %.*s\n",
                         c->lines, (int)len, start);
        if (n > 0) conn_queue(c, reply, (size_t)n);
        start = nl + 1;
    }

    size_t leftover = (size_t)(c->in + c->in_used - start);
    memmove(c->in, start, leftover);
    c->in_used = leftover;

    if (c->in_used == BUF_SIZE) {          /* a line longer than the buffer */
        conn_queue(c, "line too long\n", 14);
        c->closing = true;
    }
}

static void conn_readable(Connection *c)
{
    for (;;) {
        if (c->in_used == BUF_SIZE) break;

        ssize_t n = recv(c->fd, c->in + c->in_used,
                         BUF_SIZE - c->in_used, 0);
        if (n > 0) {
            c->in_used += (size_t)n;
            c->last_active = time(NULL);
            conn_process_input(c);
            if (c->closing) break;
            continue;                      /* drain: there may be more */
        }
        if (n == 0) {                      /* orderly close */
            c->closing = true;
            c->out_used = c->out_sent;     /* nothing left to deliver */
            break;
        }
        if (errno == EAGAIN || errno == EWOULDBLOCK) break;   /* normal */
        if (errno == EINTR) continue;
        c->closing = true;                 /* a real error */
        break;
    }
}

static void conn_writable(Connection *c)
{
    while (c->out_sent < c->out_used) {
        ssize_t n = send(c->fd, c->out + c->out_sent,
                         c->out_used - c->out_sent, MSG_NOSIGNAL);
        if (n > 0) {
            c->out_sent += (size_t)n;
            c->last_active = time(NULL);
            continue;
        }
        if (n < 0 && (errno == EAGAIN || errno == EWOULDBLOCK)) return;
        if (n < 0 && errno == EINTR) continue;
        c->closing = true;
        return;
    }
    c->out_used = c->out_sent = 0;         /* buffer drained */
}

static void accept_new(int listener)
{
    for (;;) {                             /* accept until EAGAIN */
        int fd = accept(listener, NULL, NULL);
        if (fd < 0) {
            if (errno == EAGAIN || errno == EWOULDBLOCK) return;
            if (errno == EINTR) continue;
            perror("accept");
            return;
        }
        if (!set_nonblocking(fd)) { close(fd); continue; }

        Connection *c = calloc(1, sizeof *c);
        if (c == NULL) { close(fd); continue; }
        c->fd = fd;
        c->last_active = time(NULL);
        conns[fd] = c;

        struct epoll_event ev = { .events = EPOLLIN, .data.fd = fd };
        if (epoll_ctl(epfd, EPOLL_CTL_ADD, fd, &ev) < 0) {
            close(fd); free(c); conns[fd] = NULL; continue;
        }

        conn_queue(c, "ready\n", 6);
        conn_update_interest(c);

        live++; total++;
        if (live > peak) peak = live;
    }
}

static void sweep_idle(void)
{
    time_t now = time(NULL);
    for (int fd = 0; fd < 65536; fd++) {
        Connection *c = conns[fd];
        if (c != NULL && now - c->last_active > IDLE_LIMIT) {
            conn_close(c);
        }
    }
}

int main(int argc, char *argv[])
{
    const char *port = (argc > 1) ? argv[1] : "8080";

    if (pipe(wake) < 0) { perror("pipe"); return EXIT_FAILURE; }
    set_nonblocking(wake[0]);
    set_nonblocking(wake[1]);

    struct sigaction sa;
    memset(&sa, 0, sizeof sa);
    sa.sa_handler = on_signal;
    sa.sa_flags   = SA_RESTART;
    sigaction(SIGINT,  &sa, NULL);
    sigaction(SIGTERM, &sa, NULL);
    signal(SIGPIPE, SIG_IGN);

    struct addrinfo hints;
    memset(&hints, 0, sizeof hints);
    hints.ai_family   = AF_INET6;
    hints.ai_socktype = SOCK_STREAM;
    hints.ai_flags    = AI_PASSIVE;

    struct addrinfo *list;
    int rc = getaddrinfo(NULL, port, &hints, &list);
    if (rc != 0) {
        fprintf(stderr, "getaddrinfo: %s\n", gai_strerror(rc));
        return EXIT_FAILURE;
    }

    int listener = socket(list->ai_family, list->ai_socktype,
                          list->ai_protocol);
    int yes = 1;
    setsockopt(listener, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof yes);
    if (bind(listener, list->ai_addr, list->ai_addrlen) < 0) {
        perror("bind"); return EXIT_FAILURE;
    }
    freeaddrinfo(list);

    set_nonblocking(listener);
    listen(listener, 512);

    epfd = epoll_create1(0);
    struct epoll_event ev = { .events = EPOLLIN, .data.fd = listener };
    epoll_ctl(epfd, EPOLL_CTL_ADD, listener, &ev);
    ev.data.fd = wake[0];
    epoll_ctl(epfd, EPOLL_CTL_ADD, wake[0], &ev);

    printf("event loop on port %s, one thread, Ctrl-C to stop\n", port);

    struct epoll_event events[MAX_EVENTS];
    bool running = true;
    time_t last_sweep = time(NULL);

    while (running) {
        int n = epoll_wait(epfd, events, MAX_EVENTS, 1000);
        if (n < 0) {
            if (errno == EINTR) continue;
            perror("epoll_wait");
            break;
        }

        for (int i = 0; i < n; i++) {
            int fd = events[i].data.fd;

            if (fd == listener) {
                accept_new(listener);
                continue;
            }
            if (fd == wake[0]) {           /* a signal arrived */
                char drain[64];
                while (read(wake[0], drain, sizeof drain) > 0) { }
                running = false;
                continue;
            }

            Connection *c = conns[fd];
            if (c == NULL) continue;

            if (events[i].events & (EPOLLHUP | EPOLLERR)) {
                conn_close(c);
                continue;
            }
            if (events[i].events & EPOLLIN)  conn_readable(c);
            if (events[i].events & EPOLLOUT) conn_writable(c);

            if (c->closing && c->out_sent >= c->out_used) {
                conn_close(c);              /* flush before closing */
            } else {
                conn_update_interest(c);
            }
        }

        if (time(NULL) - last_sweep >= 5) {
            sweep_idle();
            last_sweep = time(NULL);
        }
    }

    for (int fd = 0; fd < 65536; fd++) {
        if (conns[fd] != NULL) conn_close(conns[fd]);
    }
    close(listener);
    close(epfd);
    close(wake[0]);
    close(wake[1]);

    printf("\nserved %ld connections, peak concurrent %ld\n", total, peak);
    return EXIT_SUCCESS;
}

gcc -std=c17 -D_GNU_SOURCE -Wall -Wextra -g -o evserver evserver.c
./evserver 8080 &

Compare the two servers under load

# the week 40 forking server
./echoserver 8081 &

# 500 concurrent clients against each
time (for i in $(seq 500); do
        printf 'a\nb\nquit\n' | nc -q1 localhost 8081 >/dev/null &
      done; wait)

time (for i in $(seq 500); do
        printf 'a\nb\nquit\n' | nc -q1 localhost 8080 >/dev/null &
      done; wait)

Watch the process count while each runs:

ps --no-headers -C echoserver | wc -l     # hundreds
ps --no-headers -C evserver   | wc -l     # exactly 1

The forking server creates five hundred processes; the event loop stays at one thread and a few hundred bytes per connection. Raise the count to five thousand and the forking server will typically hit a resource limit while the event loop is still comfortable.

Keep many connections open and idle

for i in $(seq 1000); do (sleep 60 | nc localhost 8080 >/dev/null) & done
ps -o rss= -C evserver          # resident memory, in kilobytes
ss -tn state established '( sport = :8080 )' | wc -l

A thousand idle connections cost the event loop a few megabytes. A thread-per-connection server would have reserved eight gigabytes of stack address space for the same thing.

The two bugs this design invites

Forgetting to drain. Change conn_readable to call recv once instead of looping. Under level-triggered epoll it still works — you are told again next time — but each event does less work and throughput drops. Switch to EPOLLET with the loop removed and it breaks outright: data sits in the socket and you are never notified again. Add EPOLLET to the events and try it.

Leaving EPOLLOUT registered. Change conn_update_interest to always request EPOLLOUT and watch the CPU:

top -p $(pgrep evserver)

One connection is enough to pin a core at 100%. The socket is writable, so epoll_wait returns immediately, forever. This is the single most common event-loop bug and it looks like a mysterious performance problem rather than a logic error.

Verify the signal path

Press Ctrl-C. The handler writes one byte to the pipe; the loop sees wake[0] become readable, drains it, and exits cleanly — closing every connection and printing the statistics. Nothing unsafe happens inside the handler, which is week 39's rule honoured by construction rather than by care.

7Event loop or thread pool?

Real servers combine them: an event loop per core, each handling its own connections, with a separate pool for genuinely blocking work such as disk I/O or password hashing. nginx, Redis, and Node.js are all variations on that shape.

The "C10K problem" — handling ten thousand concurrent connections — was an open question around 2000 precisely because the thread-per-connection model could not reach it. epoll and kqueue were the answer, and the same techniques now carry C10M on commodity hardware.

8Common mistakes

10Where this leads

Week 48 attacks the parsers you have been writing — including this server's framing loop — with a fuzzer, which generates millions of inputs and finds the ones you did not think of. After that the embedded block begins, where the constraints change completely and an event loop becomes a superloop.