Fuzzing and Security Testing

1Why fuzzing works

Random input alone is nearly useless: the chance of generating a valid file header by accident is negligible, so a naive fuzzer spends its life being rejected at the first check.

Coverage-guided fuzzing changes the game. The program is instrumented so the fuzzer can see which branches an input reached. An input that reaches new code is kept and mutated further; one that does not is discarded. The result is a search that learns the input format without being told it.

input "AAAA"      → rejected at the magic check          → discard
input "CREC"      → reached the version check (new!)     → KEEP, mutate
input "CREC\x01"  → reached the length field (new!)      → KEEP, mutate
input "CREC\x01\xff\xff\xff\xff" → allocation overflow   → CRASH

Each step is a small mutation of something that worked. Within minutes a fuzzer can discover a file format that took a person an afternoon to specify — and then explore its edges far more patiently than any person would.

This is why fuzzing found thousands of bugs in OpenSSL, SQLite, and the Linux kernel, all of which had extensive test suites. Tests encode what the author expected; a fuzzer explores what the author did not.

2Writing a target

libFuzzer ships with Clang and needs one function:

#include <stdint.h>
#include <stddef.h>

int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
    parse_thing(data, size);      /* the code under test */
    return 0;                     /* non-zero is reserved */
}

clang -g -O1 -fsanitize=fuzzer,address,undefined \
      -o fuzz_parser fuzz_parser.c parser.c
./fuzz_parser corpus/ -max_len=4096

That single command compiles with instrumentation, links the fuzzing engine, and runs. There is no main.

Avoid anything involving time, randomness, the network, or the filesystem inside the target. If the code under test needs them, stub them out for the fuzz build.

The seed corpus

Start the fuzzer with valid examples rather than nothing:

mkdir corpus
printf 'timeout = 30\nname = server\n' > corpus/valid.conf
printf '# comment only\n'              > corpus/comment.conf
printf ''                              > corpus/empty.conf

Seeds save the fuzzer from rediscovering the format. Include the edge cases you already know — empty, maximum size, each optional field present and absent. A dictionary helps too, giving the mutator the keywords to try:

# parser.dict
kw1="timeout"
kw2="name"
kw3="="
./fuzz_parser corpus/ -dict=parser.dict

3Fuzzing finds nothing without sanitizers

This is the point people miss. A fuzzer detects crashes, hangs, and assertion failures. Most C memory bugs produce none of those — week 20's silent corruption runs to completion with a wrong answer.

Sanitizers convert those silent defects into immediate, loud failures:

Fuzzing without sanitizers finds a fraction of what is there. The fuzzer supplies the inputs; the sanitizers supply the detection. Neither is useful alone.

Assertions help too, because a fuzzer treats an abort as a finding. An assert stating an invariant turns a logic error into something the fuzzer can discover — which is an argument for building the fuzz target without NDEBUG.

4Triage and minimization

When a crash is found, libFuzzer writes the input to crash-<hash>. Reproduce it directly:

./fuzz_parser crash-a1b2c3d4              # replays that one input
xxd crash-a1b2c3d4 | head

Then shrink it, because a 4000-byte input tells you nothing:

./fuzz_parser -minimize_crash=1 -runs=100000 crash-a1b2c3d4

The minimizer repeatedly removes bytes while checking the crash still occurs — week 43's bisection applied to data rather than history. Four thousand bytes typically reduce to fewer than twenty, and the defect is usually obvious once they do.

Then make it a permanent test:

cp crash-a1b2c3d4 tests/regressions/overflow-in-length-field
# the test suite replays every file in that directory

A fuzzer finding the same bug twice is wasted machine time. The corpus of past crashes is as valuable as the code that fixed them.

5Threat modeling

Fuzzing finds memory-safety bugs. It does not tell you what an attacker wants. Four questions, answered before writing the code:

1. What is worth protecting? Credentials, user data, the ability to execute code, availability.
2. Where does untrusted input enter? Command-line arguments, environment, files, the network, and — as week 31 noted — the environment counts.
3. What can go wrong? For C specifically: memory corruption, integer overflow, injection, resource exhaustion.
4. What stops it? Validation at the boundary, bounded copies, checked arithmetic, limits on allocation.

The trust boundary is the central idea. Data crossing it must be validated exactly once, at the crossing, and everything inside may then assume it is well-formed. Validating repeatedly everywhere produces code where nobody is sure whether the check already happened.

6Worked example: fuzzing the week 23 parser

Take the configuration parser and attack it. The version below contains three defects deliberately — each one plausible, each one a real pattern.

/* parser.c — with three planted defects */
#include "parser.h"
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include <ctype.h>

#define MAX_NAME 32

struct Config {
    int    timeout;
    int    retries;
    char   name[MAX_NAME];
    int   *values;
    size_t value_count;
};

Config *config_parse(const uint8_t *data, size_t size)
{
    Config *c = calloc(1, sizeof *c);
    if (c == NULL) return NULL;

    char *text = malloc(size + 1);
    if (text == NULL) { free(c); return NULL; }
    memcpy(text, data, size);
    text[size] = '\0';

    char *save = NULL;
    for (char *line = strtok_r(text, "\n", &save);
         line != NULL;
         line = strtok_r(NULL, "\n", &save)) {

        if (line[0] == '#' || line[0] == '\0') continue;

        char *eq = strchr(line, '=');
        if (eq == NULL) continue;
        *eq = '\0';
        char *key   = line;
        char *value = eq + 1;

        while (*key == ' ')   key++;
        while (*value == ' ') value++;

        if (strcmp(key, "timeout") == 0) {
            c->timeout = atoi(value);                 /* DEFECT 1 */
        } else if (strcmp(key, "name") == 0) {
            strcpy(c->name, value);                   /* DEFECT 2 */
        } else if (strcmp(key, "values") == 0) {
            size_t n = (size_t)atoi(value);
            c->values = malloc(n * sizeof *c->values); /* DEFECT 3 */
            c->value_count = n;
            for (size_t i = 0; i < n; i++) c->values[i] = 0;
        }
    }

    free(text);
    return c;
}

void config_free(Config *c)
{
    if (c == NULL) return;
    free(c->values);
    free(c);
}

The fuzz target

/* fuzz_parser.c */
#include "parser.h"
#include <stdint.h>
#include <stddef.h>

int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
    if (size > 4096) return 0;              /* keep iterations fast */

    Config *c = config_parse(data, size);
    config_free(c);                          /* free, or the leak checker fires */
    return 0;
}

clang -g -O1 -fsanitize=fuzzer,address,undefined \
      -o fuzz_parser fuzz_parser.c parser.c

mkdir -p corpus
printf 'timeout = 30\nname = server\n' > corpus/valid
printf '# comment\n'                   > corpus/comment
printf 'values = 4\n'                  > corpus/values

./fuzz_parser corpus/ -max_len=4096 -print_final_stats=1

What it finds, and how quickly

Within seconds — the stack overflow.

==12345==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffd…
WRITE of size 45 at 0x7ffd… thread T0
    #0 0x… in strcpy
    #1 0x… in config_parse parser.c:44

Address is located in stack of thread T0 at offset 32 in frame
  'c' … 'name' (line 12) <== overflows this variable
artifact_written: crash-3f2a1b…

strcpy into char name[32] with no bound — week 14's defect, found in the time it takes to generate a long line. It is a classic stack smash: on a real target the return address is next.

Within a minute — the allocation overflow.

==12346==ERROR: AddressSanitizer: requested allocation size 0x…
exceeds maximum supported size

Or, worse on some platforms, no error at all: atoi returns a negative number, the conversion to size_t makes it enormous, the multiplication wraps, malloc succeeds with a tiny block, and the loop writes past it. Week 36's integer-overflow-to-heap-overflow chain, discovered automatically.

The silent one. atoi cannot report failure, so timeout = abc becomes 0. No crash, so the fuzzer is blind to it — which is the limitation worth naming. Add an assertion and it becomes findable:

assert(c->timeout >= 0 && c->timeout <= 3600);

Minimize each crash

./fuzz_parser -minimize_crash=1 -runs=100000 crash-3f2a1b…
xxd minimized-from-3f2a1b…

00000000: 6e61 6d65 203d 2041 4141 4141 4141 4141  name = AAAAAAAAA
00000010: 4141 4141 4141 4141 4141 4141 4141 4141  AAAAAAAAAAAAAAAA
00000020: 4141 0a                                  AA.

Thirty-five bytes, and the defect is unmistakable from the input alone.

The fixed parser

if (strcmp(key, "timeout") == 0) {
    long v;
    if (!parse_long(value, &v) || v < 0 || v > 3600) {
        goto invalid;                          /* refuse, do not default */
    }
    c->timeout = (int)v;

} else if (strcmp(key, "name") == 0) {
    int n = snprintf(c->name, sizeof c->name, "%s", value);
    if (n < 0 || (size_t)n >= sizeof c->name) {
        goto invalid;                          /* reject, do not truncate */
    }

} else if (strcmp(key, "values") == 0) {
    long n;
    if (!parse_long(value, &n) || n < 0 || n > MAX_VALUES) {
        goto invalid;                          /* bound it BEFORE allocating */
    }
    c->values = calloc((size_t)n, sizeof *c->values);   /* checks overflow */
    if (c->values == NULL && n > 0) goto invalid;
    c->value_count = (size_t)n;
}

Three changes, each from an earlier week: strtol with a range check (23), snprintf with truncation detected (14), and a bound before calloc (36). Rebuild and run the fuzzer again for ten minutes — it should find nothing.

Keep the crashes as tests

mkdir -p tests/regressions
cp minimized-* tests/regressions/
# and in CI:
for f in tests/regressions/*; do ./fuzz_parser "$f" || exit 1; done

Every past crash replays on every build, in milliseconds. This is the highest-value output of a fuzzing session — higher than the fix, because the fix can be undone and the test cannot be undone quietly.

Run it continuously

# overnight, with a persistent corpus
./fuzz_parser corpus/ -max_total_time=28800 -jobs=8 -workers=8

# AFL++, a different mutation strategy — worth running both
afl-clang-fast -fsanitize=address -o parser_afl parser_afl.c parser.c
afl-fuzz -i corpus -o findings -- ./parser_afl

Fuzzing is not a one-off. Projects that take it seriously run it continuously — OSS-Fuzz does exactly this for hundreds of open-source C projects — because new code introduces new paths, and a corpus grown over months explores far deeper than one grown over minutes.

7Common mistakes

9Where this leads

Week 49 begins the embedded block. Everything changes: no operating system, no malloc, a few kilobytes of RAM, and a compiler producing code for a different processor than the one it runs on. The discipline built over the last forty-eight weeks is exactly what makes that environment tractable.