Chapter 15: Linux
Introduction
Learning Outcomes
Trace the line from UNIX through BSD, MINIX and GNU to Linux; place distributions and Android.
Separate the library / system-call / kernel layers; show how pipelines are built with fork, pipe and dup2.
Explain the kernel steps of fork and execve, signal delivery, task_struct, clone() flags and NPTL; what changed in the move from CFS to EEVDF.
Explain the address-space layout, 4/5-level page tables, memory zones, the SLUB allocator, the page cache and the OOM killer.
Device files, blk-mq, the network stack; the VFS, i-nodes, ext4 extents and journaling, links and /proc.
Compute rwx/setuid/umask; recognize capabilities, namespaces + cgroups, SELinux/AppArmor, seccomp and the Android sandbox.
Introduction and History · Step by step
Timeline from UNIX to Linux
Ken Thompson and Dennis Ritchie wrote UNIX in assembly on a PDP-7 in 1969; rewriting it in C in 1973 made it portable. AT&T gave the source to universities, and BSD was born at Berkeley.
As the commercial UNIXes (System V, SunOS, AIX…) diverged, POSIX defined a common interface. Tanenbaum wrote MINIX for teaching; in 1991 Linus Torvalds developed his own kernel on MINIX and combined it with the GNU tools.
Linux 1.0 (1994); multiprocessing with 2.0; a preemptible kernel and the O(1)/CFS schedulers with 2.6. In 2005, after the BitKeeper dispute, Torvalds wrote git. The first Android phone shipped in 2008.
Since 2017 all of the world's 500 fastest supercomputers run Linux. The 6.x series brought Rust support, EEVDF and sched_ext; version jumps carry no meaning (7.0 follows 6.19).
Introduction and History
GNU, Linux and Distributions
| Family | Examples | Package |
|---|---|---|
| Debian | Debian, Ubuntu, Mint, Pardus | .deb / apt |
| Red Hat | Fedora, RHEL, Rocky, Alma | .rpm / dnf |
| Arch | Arch, Manjaro | pacman |
| SUSE | openSUSE, SLES | zypper |
| Lightweight | Alpine (musl + BusyBox) | apk |
Release model: a major release every ~9–10 weeks (a 2-week merge window + rc1…rc7). The last release of each year becomes LTS: 6.6, 6.12, 6.18…
Strictly, “Linux” is only the kernel's name; the system users install is a distribution (GNU/Linux).
Introduction and History
Design Goals and the UNIX Philosophy
A few well-defined concepts: process, file, pipe.
Small tools that do one thing combine to do big jobs.
“Everything is a file”: disks, terminals and pipes all use the same read/write.
Shell scripts, pipelines, regular expressions.
Text streams are the universal interface; tools work together without knowing about each other.
Designed by programmers for programmers: little “magic”, silent success.
POSIX standardizes this interface: the same source compiles on Linux, BSD and macOS.
The Shell and Interfaces · Step by step
Layers and Interfaces: The Journey of a printf
The program calls printf. This is not a system call yet: glibc writes the text into a buffer in user space.
When the buffer is flushed, glibc's write() wrapper runs: it puts the system-call number (write = 1 on x86-64) in rax and the arguments in registers, and executes the syscall instruction.
The CPU switches to kernel mode. The entry code uses the number to pick the handler from sys_call_table; the VFS finds the object behind file descriptor 1.
fd 1 is attached to a pseudoterminal (pty); the tty layer passes the data to the terminal emulator. Had it been a disk, the path would go to the page cache and the block layer.
The return value comes back in rax (8 bytes), and sysret returns to user mode. On error, glibc turns the negative value into errno and returns −1.
The Shell and Interfaces
Basic Commands and Redirection
| Category | Commands | Note |
|---|---|---|
| Files / directories | ls cd pwd mkdir rm -r cp mv touch find tar | touch updates the timestamp; creates an empty file if none exists |
| Text (filters) | cat less head tail grep sort uniq wc echo | read stdin, write stdout: they combine in pipelines |
| Processes / system | ps top kill chmod chown df du sync | sync flushes dirty buffers to disk (syncing between machines: rsync) |
| Network | ssh scp ping curl wget traceroute | ip addr, ip route, ss, nft; they replaced the old ifconfig, route, netstat and iptables |
| Information | man uname free lsblk | /proc is not a command but a pseudo file system: cat /proc/meminfo |
# redirection: 0 = stdin, 1 = stdout, 2 = stderr sort < input.txt > output.txt 2> errors.log # < reads, > writes (overwrite) echo "line" >> log.txt # >> appends make 2>&1 | tee build.log & # stderr→stdout, pipe, background
The Shell and Interfaces · Step by step
Pipelines: Data Flows Through the Stages
The shell parses the line and creates 3 pipes and 4 processes. The processes run at the same time; each knows only its own stdin/stdout.
cat writes the file's 7 lines into the pipe. If the pipe buffer (64 KiB by default) fills, the writer blocks: producer–consumer.
grep reads line by line and passes the 5 matching lines on at once. A filter is stateless; it does not hold up the flow.
sort, however, cannot output anything before reading all of its input (up to EOF): the last line could change the first position.
uniq -c counts only adjacent repeats, which is why sort comes first. Result: 2 × “error: net”, 3 × “error: disk”.
The Shell and Interfaces · Step through the code
How Does the Shell Build a Pipeline?
int fd[2]; pipe(fd); /* 3: read, 4: write */ if (fork() == 0) { /* child 1 */ dup2(fd[1], 1); close(fd[0]); close(fd[1]); execlp("ls", "ls", NULL); } if (fork() == 0) { /* child 2 */ dup2(fd[0], 0); close(fd[0]); close(fd[1]); execlp("wc", "wc", "-l", NULL); } /* shell: close both ends */ close(fd[0]); close(fd[1]); wait(NULL); wait(NULL);
pipe() creates a buffer in the kernel and two ends: the lowest free numbers, 3 (read) and 4 (write).
fork(): the child gets a copy of the descriptor table; both processes point to the same pipe.
dup2(4, 1): stdout is now the pipe's write end. The extra ends are closed.
execlp replaces the image with ls; the descriptor table is kept. ls writes into the pipe without knowing it.
The second child connects its stdin to the pipe's read end and becomes wc -l.
The shell closes both ends. Otherwise the write end would stay open, and wc would never see EOF and wait forever. Then it waits for both children.
Kernel Structure
Structure of the Linux Kernel
The system-call interface on top, interrupts and the dispatcher (context switching) at the bottom. All components run in a single address space: Linux is a monolithic kernel. The classic 4.4BSD kernel split into the same three components.
Kernel Structure · Step by step
Loadable Kernel Modules (LKMs)
Monolithic but modular: most drivers are loadable kernel modules (.ko). The administrator requests one with modprobe, or udev loads it automatically when the device appears.
modprobe finds the dependencies in /lib/modules/$(uname -r)/modules.dep and loads them first (insmod does not).
The finit_module() system call copies the module into the kernel; with Secure Boot on, its signature is checked. It requires the CAP_SYS_MODULE capability.
The kernel applies the module's relocations and binds the symbols it uses to kernel functions exported with EXPORT_SYMBOL.
The module_init function runs: the driver registers itself with the PCI subsystem (pci_register_driver).
probe() is called for the matching device and the network interface appears. lsmod lists modules, rmmod removes one. A module runs with kernel privilege: its bug can crash the whole system.
The source tree is ~40 million lines, mostly drivers. The kernel has had Rust support since 6.1, and the first Rust driver arrived in 6.8; eBPF runs small, safe, verified programs in the kernel.
Processes
What Do fork and exec Do in the Kernel?
fork() in the kernel
- Allocate a new
task_struct, fill it from the parent, assign a new PID - Allocate a kernel stack; copy registers (return value 0 in the child)
- Copy the page tables and mark writable pages read-only (COW)
- Share open files: the
struct filereference count goes up - Signal handlers, cwd and credentials are copied
execve() in the kernel
- The ELF header is read; the old
mm_structis dropped, a new one is built and the segments aremmapped - A new
credis prepared according to the setuid/setgid bits - PC ←
ld.so(the dynamic linker) or the program's entry point - A failed
execvereturns −1; by convention the child exits with_exit(127)
Old UNIX really copied data and stack on fork; Linux copies only the page tables (COW). Since exec usually follows fork at once, very few pages are ever copied.
Processes
POSIX Signals
A signal is a software interrupt that notifies a process of an event. It can be synchronous (SIGSEGV, SIGFPE: caused by the process's own instruction) or asynchronous (SIGINT, SIGTERM: from outside). For each signal a process chooses the default action, ignoring it, or a handler (cleanup, logging).
| Signal | No | When is it raised? | Default |
|---|---|---|---|
| SIGHUP | 1 | The terminal hung up; for daemons, “reload your configuration” | terminate |
| SIGINT | 2 | Ctrl-C (to the foreground process group) | terminate |
| SIGQUIT | 3 | Ctrl-\ | core dump |
| SIGILL | 4 | Illegal machine instruction | core dump |
| SIGABRT | 6 | abort(): the process terminates itself abnormally | core dump |
| SIGFPE | 8 | Arithmetic error such as integer division by zero | core dump |
| SIGKILL | 9 | kill -9; cannot be caught or ignored | terminate |
| SIGSEGV | 11 | Invalid memory access | core dump |
| SIGPIPE | 13 | Writing to a pipe with no readers | terminate |
| SIGALRM | 14 | alarm() timer expired | terminate |
| SIGTERM | 15 | The default of the kill command; a polite request to terminate | terminate |
| SIGUSR1/2 | 10/12 | User-defined | terminate |
| SIGCHLD | 17 | A child stopped or terminated | ignore |
| SIGSTOP / SIGCONT | 19/18 | Stop (uncatchable) / continue; Ctrl-Z → SIGTSTP | stop / continue |
Numbers are for x86/ARM Linux. SIGKILL and SIGTERM can both be sent with the kill command; the difference is whether they can be caught.
Processes · Step by step
Signal Delivery
The user presses Ctrl-C: the tty line discipline raises SIGINT for the foreground process group. The kernel merely sets bit 2 in the target task's pending signal set.
The signal is not handled at once. When the process returns to user mode from a system call or an interrupt, the kernel checks for signals that are pending and unmasked.
If a handler was installed with sigaction, the kernel writes a signal frame (saved registers + a return trampoline) onto the user stack and sets the PC to the handler.
The handler runs in user mode. Only async-signal-safe functions are safe inside it (e.g. write, a volatile sig_atomic_t flag); printf and malloc are not.
When the handler returns, the trampoline makes the rt_sigreturn system call and the kernel restores the registers from the frame. (sigreturn does not enter the handler; it returns from it.)
The process resumes at the interrupted instruction. Without a handler the default action would apply: termination for SIGINT.
Processes · Step through the code
Choosing Pending Signals: Code Trace
/* on return to user mode (kernel) */ void check_signals(task *t) { unsigned long h = t->pend & ~t->blk; while (h) { int sig = __builtin_ctzl(h) + 1; t->pend &= ~(1UL << (sig - 1)); deliver(t, sig); /* handler/default */ h = t->pend & ~t->blk; } } /* before: sigprocmask(SIG_BLOCK, {SIGINT}) */ /* after: sigprocmask(SIG_UNBLOCK, {SIGINT}) */
Before entering a critical section the process blocked SIGINT with sigprocmask (bit 2 of the blocked set). Signals are a bit set in each task.
Three signals arrive while the process sleeps: Ctrl-C (2), kill -USR1 (10) and a child's exit (17). The kernel only sets bits in the pending set; nothing runs yet.
As the process returns to user mode the kernel checks: ready = pending & ~blocked. SIGINT is pending but blocked; ready to deliver are 10 and 17.
The lowest ready bit (ctz): 10. It is cleared from pending and delivered: SIGUSR1 → the handler runs. The ready set is recomputed.
The lowest ready bit (ctz): 17. It is cleared from pending and delivered: SIGCHLD → default: ignore. The ready set is recomputed.
The ready set is empty, the loop ends and the process continues in user mode. SIGINT is not lost: it stays in the pending set.
The critical section is over and SIGINT is unblocked. The same check runs on return from sigprocmask: ready = {2}.
SIGINT is delivered; with no handler installed the default action applies: the process terminates. Blocking a signal defers it; it does not ignore it. Had the same signal arrived several times, it would have merged into one bit.
Processes
Process-Management System Calls
| System call | Description |
|---|---|
fork() | Creates a new process by copying the caller (COW) |
clone(), clone3() | A new task with flags choosing what is shared (threads, containers) |
execve() | Replaces the current process image with a new program |
waitpid() | Waits for a child to finish and gets its exit status (reaps the zombie) |
exit() / _exit() | Terminates the caller (exit flushes stdio buffers) |
kill() | Sends a signal to a process; the signal need not mean termination |
sigaction() | Sets the handler or action for a signal |
sigprocmask() | Reads/changes the signal mask (blocked signals) |
sigpending() | Returns the signals pending because they are blocked |
sigsuspend() | Temporarily changes the mask and sleeps until a signal arrives |
rt_sigreturn() | Restores the saved context on return from a handler |
alarm() / pause() | SIGALRM after n seconds / sleep until a signal arrives |
A system call is the only way for a user program to ask the kernel for service; Linux x86-64 has ~460 of them. The process tree is shown by pstree -p or ps -ef --forest.
Processes
The Process Descriptor: task_struct
In Linux there is no structural difference between a process and a thread: both are a task (task_struct). The difference is which substructures they share. The PID users see is really the tgid (thread group ID). With __state D (uninterruptible sleep) a task does not die at once even on SIGKILL, which is why most waits now use TASK_KILLABLE.
Processes · Step by step
clone() and Threads
In glibc, fork() is a clone() called with just the SIGCHLD flag: the child gets a copy of every structure (memory via COW). Its own PID and TGID: a new process.
pthread_create() → clone(CLONE_VM | CLONE_FS | CLONE_FILES | CLONE_SIGHAND | CLONE_THREAD | …). Everything is shared; the new task joins the same TGID. NPTL: every thread is a kernel task (the 1:1 model).
Namespace flags (CLONE_NEWPID, CLONE_NEWNS, CLONE_NEWNET…) give the child a new view: inside, it believes it is PID 1. This is the basis of containers.
Scheduling and Booting
From CFS to EEVDF: Fair Sharing
| Version | Selection rule of the fair class | Data structure |
|---|---|---|
| 2.4 (2001) | O(n): every pick scans all ready tasks | a single list |
| 2.6.0–2.6.22 (2003) | O(1): a list per priority, active and expired arrays | array of 140 lists + bitmap |
| 2.6.23–6.5 (2007) | CFS: smallest vruntime | red-black tree |
| 6.6+ (2023) | EEVDF: earliest virtual deadline among eligible tasks (v ≤ V) | the same tree, augmented with subtree data |
- Unchanged: the vruntime in
sched_entity, the nice → weight table (sched_prio_to_weight, nice 0 → 1024) and the tree in the run queue. - CFS's latency knobs (
sched_latency, wakeup granularity) gave way to a single base slice (base_slice_ns); the virtual deadline is d = v + slice × 1024 / weight. - A latency-sensitive task can ask for a shorter slice with
sched_setattr()(6.12+): a short slice means an early deadline and being picked more often.
Scheduling and Booting · Step by step
Booting: From UEFI to systemd
At power-on the UEFI firmware tests the hardware and loads the EFI application from the ESP partition. With Secure Boot on, each stage verifies the next one's signature.
The bootloader loads the kernel image (vmlinuz) and the initramfs into memory and passes the command-line parameters. The kernel can also be started directly as an EFI application (UKI).
The kernel decompresses itself, and start_kernel() sets up memory management, interrupts and the scheduler. The first task becomes PID 0, the idle task; kernel_init (PID 1) and kthreadd (PID 2) are created.
The drivers needed to mount the root file system may not be built into the kernel (NVMe, RAID, encrypted disks). The initramfs provides a temporary root in RAM, mounts the real root and does a switch_root.
PID 1 is now systemd: it starts services in parallel according to the dependencies in the unit files and reaches the target (multi-user.target or graphical.target).
On the console agetty → login → shell; on the desktop gdm; remotely sshd. The classic UNIX chain init → getty → login → sh is still recognizable.
Memory Management
The Virtual Address Space of a Process
Each region is described by a VMA (vm_area_struct); VMAs are kept in a “maple tree” inside the mm_struct. The stack and mmap grow down and the heap grows up; the gap between them is only virtual and costs no physical memory.
Memory Management
Sharing, Mapped Files and Memory Calls
| Call | Function |
|---|---|
brk / sbrk | Grows/shrinks the heap (data segment) by moving its end |
mmap | Maps a file, a device or anonymous memory into the address space |
munmap | Removes a mapping and returns the memory to the system |
mprotect | Changes a region's permissions (r/w/x) |
madvise | Gives the kernel a usage hint (e.g. MADV_DONTNEED) |
msync | Writes a shared mapping's dirty pages to the file |
MAP_SHARED: writes are visible in the file and to other processes. MAP_PRIVATE: copy on write; the file does not change. glibc malloc serves small requests from the heap and requests ≥ 128 KiB directly with mmap.
Memory Management · Step by step
Page-Table Walk: 4 and 5 Levels
On x86-64 a 4 KiB page → a 12-bit offset. The remaining 36 bits are split across four tables of 512 entries (9 bits) each. Bits 63–48 copy bit 47 (a canonical address).
The CR3 register points to the process's top-level table (PGD); CR3 changes on a context switch. Index 254 → the address of the PUD table.
PUD[232] points to a PMD table. (A 1 GiB huge page could also end here.)
PMD[225] points to a PTE table. (A 2 MiB huge page / THP ends here.)
PTE[181]: the present bit, frame number, and R/W, U/S, NX, dirty (D) and accessed (A) bits. Present = 0 means a page fault.
The frame number is joined with the offset: physical address = frame × 4096 + offset.
Four extra reads on every access are expensive: the TLB caches the translation. Five-level paging (LA57, Linux 4.14+) inserts a P4D level for 57-bit addresses. Linux's generic model has 5 levels; when the hardware uses fewer, levels are folded.
Memory Management
Representing Physical Memory
Linux describes physical memory as a node → zone → page hierarchy. On a multisocket machine each CPU prefers to allocate from its own node (NUMA). 64-bit systems have no ZONE_HIGHMEM; it survives only in 32-bit kernels. Each zone's free_area array holds the buddy allocator's free lists for orders 0–10 (4 KiB–4 MiB); see /proc/buddyinfo.
Memory Management
The SLUB Object Allocator
The kernel allocates many small objects of the same size (task_struct, dentry, inode); allocating a page for each would be wasteful. SLUB keeps object caches; kmalloc uses size classes from 8 B to 8 KiB. SLOB (6.4) and SLAB (6.8) were removed: SLUB is the only allocator. For large areas that need not be physically contiguous, vmalloc.
Memory Management · Step through the code
The SLUB Free List: Code Trace
void *slab_alloc(kmem_cache *c) { void *obj = c->freelist; /* head of list */ if (!obj) obj = new_slab(c); /* slow path */ c->freelist = *(void **)obj; /* next */ return obj; } void slab_free(kmem_cache *c, void *obj) { *(void **)obj = c->freelist; /* inside the object */ c->freelist = obj; } /* a,b = alloc; free(a); c,d,e,f = alloc */
A new slab: a 4 KiB page cut into four 1 KiB objects. Free objects form a list; the “next” pointer is stored not in a separate structure but inside the free object itself.
slab_alloc: the head of the list (+0) is taken, and freelist advances to the next pointer stored in it (+1024). No lock, no page allocation: a few instructions.
b gets +1024; freelist points to +2048.
a is freed: the old head (+2048) is written into the object's first 8 bytes and freelist becomes +0. The list works like a stack (last in, first out).
c takes the +0 that a just released: most likely still in the CPU cache, so a “hot” object.
d takes +2048 and e takes +3072. The next pointer inside +3072 was NULL: the slab is full and freelist is empty.
For f the list is empty: the slow path. A new page comes from the page (buddy) allocator, is cut into four objects, and the first goes to f. The common case fast, the rare case correct.
Memory Management · Step by step
Page Cache, Reclaim and OOM
Free memory is wasted memory: Linux uses unused RAM as the page cache for file data. “available” in free -h also counts the reclaimable part of this cache.
The application grew and free memory dropped below the low watermark: kswapd wakes and frees clean cache pages in the background (dirty ones are written to disk first).
MGLRU (6.1) sorts pages into access generations; the coldest generation goes first. Cold anonymous pages have no file: zswap compresses them in RAM and, if needed, writes them to swap.
Allocation outpaces reclaim and free memory is below the min watermark: the allocating process does direct reclaim itself and stalls (latency rises).
If nothing else works, the OOM killer kills the process with the highest oom_score (usually the largest) with SIGKILL; oom_score_adj can influence the choice. If a cgroup limit is exceeded, the victim is picked from that group only.
Input/Output
Devices Are Files: Major and Minor Numbers
$ ls -l /dev # selected lines
brw-rw---- 1 root disk 8, 0 sda
brw-rw---- 1 root disk 8, 1 sda1
brw-rw---- 1 root disk 259, 0 nvme0n1
crw-rw-rw- 1 root root 1, 3 null
crw-rw-rw- 1 root tty 5, 0 tty- c character device: byte stream, usually uncached (terminals,
/dev/null, mice) - b block device: fixed-size blocks, random access; uses the page cache and I/O scheduler
- Network interfaces (
eth0) aren't in/dev; they're used via sockets - devtmpfs + udev create nodes dynamically as devices appear
“major, minor” replaces the size column. The major number selects the driver, the minor the unit it manages. Applications reach disks and terminals with the same open/read/write.
Input/Output
Structure of the Linux I/O System
The three device types go down three separate paths. Classic BSD had a similar split; System V's STREAMS module stack never entered the mainline Linux kernel. On the network path data travels in sk_buff structures; XDP/eBPF can process packets at the driver level.
Input/Output
The Driver Interface: file_operations and Terminals
static const struct file_operations my_fops = { .owner = THIS_MODULE, .open = my_open, .release = my_release, /* close */ .read = my_read, .write = my_write, .unlocked_ioctl = my_ioctl, .poll = my_poll, .mmap = my_mmap, }; cdev_init(&cdev, &my_fops); /* major:minor → fops */
| Old UNIX entries | Linux equivalent |
|---|---|
k_open, tty_open, pr_open, ip_open | .open |
k_close, tty_close, pr_close… | .release |
mem_read, k_read, tty_read | .read |
mem_write, tty_write, pr_write… | .write |
k_ioctl, tty_ioctl, pr_ioctl… | .unlocked_ioctl |
tcgetattr/tcsetattr read and write terminal parameters: canonical mode (line editing, ICANON) or raw mode, ECHO. cfset/cfget[io]speed set the input/output baud rate; today mostly for serial ports, since terminals are pseudoterminals (pty).The classic textbook table lists separate entry points per driver (k_*, tty_*, pr_*, ip_*); Linux gathers them into one structure of function pointers. The VFS routes every read() to the driver's .read.
Input/Output
The Block Layer: blk-mq and I/O Schedulers
The Linux block layer blk-mq is multiqueue: each CPU writes to its own software queue, and requests pass through an optional I/O scheduler (none, mq-deadline, bfq, kyber) before being spread over the device's hardware queues. The active scheduler is read and changed per device in /sys/block/<device>/queue/scheduler.
Input/Output · Step by step
Networking: Sockets and Connection Setup
The server creates a socket (fd 3), binds it to port 80 (bind) and opens a listen queue with listen. A socket is a file descriptor too.
The server blocks in accept. The client creates its own socket.
With connect, the client kernel performs the three-way handshake (SYN, SYN-ACK, ACK); the application never sees these packets.
Once connected, accept returns a new socket (fd 4); fd 3 keeps listening. Servers with many clients manage thousands of sockets with epoll or threads.
Data flows with write/read: the kernel puts it in sk_buffs, adds TCP/IP headers, and the driver writes it to the card's ring. close sends a FIN.
File Systems
The Directory Tree and Mounting
/usr/bin: commands (/binis now a link to it)/etc: text configuration files/dev: device files/usr/lib: shared libraries,/usr/lib/modules: kernel modules/usr: read-only, shareable software (bin, lib, share)/home: user directories;/root: the administrator's/var: variable data (logs, spools, caches)/proc,/sys: the kernel's pseudo file systems/boot: the kernel and initramfs/tmp: temporary (tmpfs on most distributions)
Instead of Windows-style drive letters there is a single tree: another file system is mounted on any directory. The layout follows the FHS standard; most distributions moved /bin and /lib under /usr (“usrmerge”).
File Systems
The VFS: Descriptors, Open Files and i-nodes
In Linux an open file is a struct file (f_pos, flags, reference count); the per-process table lives in task_struct → files_struct, and after fork() both processes point to the same struct file. The objects' operation tables (super_operations, inode_operations, file_operations) route each call to ext4's, XFS's, Btrfs's, NFS's or procfs's own function; dentries are kept in the dcache.
File Systems
File and Directory System Calls, stat
| File call | Function |
|---|---|
open, creat | Open / create, return an fd |
close | Close the descriptor |
read, write | Read / write at the position |
lseek | Move the position |
stat, fstat, lstat | Get information (lstat: the link itself) |
pipe, dup2 | Create a pipe / duplicate an fd |
fcntl | Locks, flags, etc. |
mkdir, rmdir | Create / remove a directory (if empty) |
link, unlink | Add / remove a name |
rename, chdir | Move (atomically) / change cwd |
getdents64 | Read directory entries |
| stat field | Meaning |
|---|---|
st_dev | Device the file resides on |
st_ino | i-node number |
st_mode | Type + permission bits |
st_nlink | Number of hard links |
st_uid, st_gid | Owner and group |
st_size | Size in bytes |
st_atime | Last access time |
st_mtime | Last content change |
st_ctime | Last i-node change (not creation!) |
st_blocks | Number of 512 B blocks allocated |
opendir/readdir/closedir are not system calls but library functions built on getdents64. The creation time is available only through the newer statx() (stx_btime).
File Systems · Step by step
File Locking
Process A puts a shared (read) lock on bytes 4–7: fcntl(fd, F_SETLK, &fl) with l_type = F_RDLCK, l_start = 4, l_len = 4.
B asks for a shared lock on 6–9: shared locks may overlap freely, so the request is granted.
C puts a shared lock on 2–11; bytes 6–7 are now covered by three locks.
D asks for an exclusive (F_WRLCK) lock on byte 9: it conflicts with B's and C's locks. F_SETLK fails at once; F_SETLKW waits until the locks go away. If the kernel detects a wait cycle it returns EDEADLK.
These locks are advisory: a process that never asks for the lock can still write. For whole files there is flock(); for thread-safe use, OFD locks bound to the open file.
File Systems
ext2/ext4 Disk Layout and the i-node
ext2 split the classic UNIX layout (boot block, superblock, i-nodes, data) into block groups: an i-node and its data are kept close in the same group, and the superblock is backed up. With indirect blocks the theoretical limit is ~4 TiB (2 TiB in ext2 because of the i_blocks field). The root directory's i-node number is always 2.
File Systems · Step by step
ext4 Journaling
One write() + fsync() changes several blocks: the data (D) plus metadata such as the i-node, the block bitmap and the extent. A crash after writing one but not another leaves the file system inconsistent.
In the default data=ordered mode the data blocks are written to their home locations first, so metadata never points at a block of garbage.
The metadata blocks are written to the journal as one transaction (T42), ending with a commit record. Once the commit reaches the disk, the transaction is durable.
Checkpoint: the metadata is written to its home locations and the journal space is reused (a circular buffer).
After a crash, instead of a full fsck scan the journal is read: committed transactions are replayed, incomplete ones discarded. Consistency in seconds.
Other modes: data=journal (data goes to the journal too; safest, slowest) and data=writeback (no ordering guarantee). XFS also journals metadata; Btrfs stays consistent with copy on write instead of a journal.
File Systems · Step by step
Directories and Path Resolution
An absolute path starts at the root: the root directory's i-node (2) is read and its data blocks are searched for “home” → 131073. The x (search) permission is checked for every directory.
“ali” is looked up in the data of the “home” directory → 131074.
“notes.txt” in the “ali” directory → 132010. (A relative path would start at the process's cwd.)
The last component's i-node is read, permissions are checked, and a struct file is created and an fd returned.
If every step needed a disk read it would be slow: the dentry cache (dcache) keeps name → i-node mappings in memory, so hot paths resolve without touching the disk.
File Systems
Hard and Symbolic Links
$ ln a.txt backup/b.txt # hard $ ln -s a.txt s # symbolic $ ls -li 500 -rw-r--r-- 2 ali ali 9 a.txt 777 lrwxrwxrwx 1 ali ali 5 s -> a.txt
- Hard link: the same i-node, and the link count goes up.
rm a.txtremoves only the name; the data is freed when the count reaches 0 (and no open fd remains). - A hard link cannot cross file-system boundaries or point to a directory.
- Symbolic link: holds the target's path; it can point to another file system or a directory. If the target is deleted, the link dangles.
File Systems · Step through the code
Link Count and unlink: Code Trace
int fd = open("a.txt", O_CREAT|O_RDWR, 0644); write(fd, "data", 4); link("a.txt", "b.txt"); /* hard link */ symlink("a.txt", "s.txt"); /* symbolic */ unlink("a.txt"); int r1 = open("b.txt", O_RDONLY); int r2 = open("s.txt", O_RDONLY); unlink("b.txt"); close(r1); close(fd);
open allocates a new i-node (812) and adds the entry “a.txt → 812” to the directory: link count 1. fd 3 is open: open-file count 1.
link only writes a new directory entry: b.txt → 812. No data is copied; the link count is 2. a.txt and b.txt are equals; neither is the “original”.
symlink creates a separate i-node (813) whose contents are just the path “a.txt”. 812's link count does not change.
unlink("a.txt") removes the name and drops the link count to 1. The data is still reachable through b.txt.
b.txt opens fine: r1 = 4, open-file count 2.
Opening s.txt, the kernel follows the path inside it (“a.txt”); that name is gone. The symbolic link dangles: r2 = −1, ENOENT.
The last name is gone too: link count 0. But two descriptors are open, so the i-node and data are not freed yet; ls no longer shows the file.
close(r1): open-file count 1. Reads and writes through fd 3 still work.
The last descriptor is closed: both counters are 0, so the i-node and data blocks are freed. The “open, then unlink at once” idiom for temporary files relies on this.
File Systems
Beyond ext4: XFS, Btrfs and Others
| File system | Key feature | Typical use |
|---|---|---|
| ext4 | Journaling, extents, mature with a fast fsck | Default on most distributions |
| XFS | Parallel I/O, large files, delayed allocation | RHEL default, large servers |
| Btrfs | Copy on write, snapshots, checksums, compression, subvolumes | Fedora, openSUSE default |
| F2FS | Flash-friendly, log-structured | Android phones |
| tmpfs | Entirely in RAM (page cache + swap) | /tmp, /run, /dev/shm |
| NFS, SMB | Network file systems | Shared home directories |
Thanks to the VFS they all work with the same system calls; findmnt or df -T shows which file system is mounted where.
File Systems
/proc and /sys: Windows into the Kernel
$ cat /proc/loadavg
0.42 0.37 0.30 2/1234 98765
$ grep -E "MemTotal|MemAvailable" /proc/meminfo
MemTotal: 16303452 kB
MemAvailable: 9876544 kB
$ ls /proc/self
cmdline cwd environ exe fd maps status …
$ cat /sys/block/nvme0n1/queue/scheduler
[none] mq-deadline kyber bfq
$ sysctl vm.swappiness # = /proc/sys/vm/swappiness
vm.swappiness = 60Per-process directories (/proc/PID/status, maps, fd/) and kernel statistics. ps, top and free read their data here. /proc/sys: tunable kernel parameters.
The device model: every device, driver and bus is a directory; one value per file. /sys/fs/cgroup is the cgroup v2 interface.
Security · Step by step
chmod and umask Arithmetic
Each octal digit is three bits: 7 = 111 = rwx, 5 = 101 = r-x, 4 = 100 = r--.
Symbolic mode changes the existing bits: the group write bit is added and the others' read bit removed → 0770.
A new file asks for 666 and a directory for 777; the bits in umask are cleared: 666 & ~022 = 0644, directory 777 & ~022 = 0755. (Bit clearing, not subtraction!)
A stricter umask: file 666 & ~027 = 0640 (rw-r-----), directory 777 & ~027 = 0750 (rwxr-x---). Others see nothing.
4xxx setuid: the running process takes the file owner's identity → rwsr-xr-x. 2xxx setgid (on a directory: new files inherit the group). 1xxx sticky bit: everyone writes in /tmp, but a file can be deleted or renamed only by the file's owner, the directory's owner or root (a suitably privileged process) → rwxrwxrwt.
Check: stat -c "%a %A" file shows the octal and symbolic modes together; umask -S prints the mask symbolically.
Security
Access Control, root and Capabilities
- euid = 0 (root) → allow (execution needs at least one x bit)
- euid = file owner → look only at the owner bits
- egid or a supplementary group = file group → group bits
- otherwise the other bits
The first matching class wins: for a ----rw-rw- file, its owner ali cannot read it while others can.
setcap cap_net_raw+ep /usr/bin/ping, ping opens a raw socket without being setuid root.| Call | Function |
|---|---|
chmod | Changes a file's permission mode |
chown | Changes owner and group |
access | Tests access with the real uid/gid |
getuid, geteuid | Real / effective uid |
getgid, getegid | Real / effective gid |
setuid, setresuid | Changes the uid (dropping privilege) |
setgid | Changes the gid |
umask | Permission mask for new files |
User and group IDs live in /etc/passwd and /etc/group; password hashes are kept in /etc/shadow, readable only by root. sudo grants authorized users root rights per command.
Security · Step through the code
File Permission Check: Code Trace
int has_perm(cred *c, inode *f, int req) { int bit, m = f->mode; /* r4 w2 x1 */ if (c->euid == 0) /* root */ return !(req & 1) || (m & 0111); if (c->euid == f->uid) bit = (m >> 6) & 7; /* owner */ else if (in_group(c, f->gid)) bit = (m >> 3) & 7; /* group */ else bit = m & 7; /* other */ return (bit & req) == req; }
report.txt: owner ali, group team, mode 0640 = rw-r-----. veli is in the team group; can is not.
ali owns the file: the owner bits are chosen, bit = 6 (rw-). 6 & 2 == 2: allowed.
veli is in the team group: the group bits are chosen, bit = 4 (r--). 4 & 2 ≠ 2: denied.
veli is in the team group: the group bits are chosen, bit = 4 (r--). 4 & 4 == 4: allowed.
can is neither the owner nor in the group: the other bits are chosen, bit = 0 (---). 0 & 4 ≠ 4: denied.
root (euid 0): for reading and writing the permission bits are not checked at all; allowed. Only execution requires at least one x bit.
Take mode 0066 (----rw-rw-): can and veli may read, but ali falls into the owner class, which has no bits: denied. The first matching class wins; the other bits are never consulted.
Security · Step by step
Containers: Namespaces + cgroups
The runtime (runc, crun) creates new namespaces with clone()/unshare(): its own PID numbers, mount tree, network stack, host name, IPC, user mapping (root inside is an ordinary uid outside), cgroup and clock views.
Inside the mount namespace, pivot_root switches to the image's root file system. The root is usually overlayfs: read-only image layers + a writable upper layer private to the container.
cgroup v2 limits resources: cpu.max = "50000 100000" (half a CPU), memory.max = 512M, pids.max = 100. Namespaces decide what is seen; cgroups decide how much is used.
The attack surface is shrunk: unneeded capabilities are dropped, and no_new_privs, a seccomp filter (dozens of risky system calls blocked) and an AppArmor/SELinux profile are applied.
Finally execve: the application is PID 1 inside and PID 48213 on the host. All share the same kernel: much lighter than a VM, but a kernel bug affects every container.
Security
seccomp and Linux Security Modules (LSM)
Label-based mandatory access control (MAC): every process and file has a type, and the policy says what each type may do to which. RHEL, Fedora, Android.
Path-based profiles: “this program may access only these files”. Easy to write. Ubuntu, SUSE, Debian.
An unprivileged process sandboxes itself: “from now on I may write only under ./data”.
Filters the system calls a process may make; used by Chrome, systemd services, Docker and Android apps.
Android and Summary · Step by step
Android: Binder, Zygote and the Sandbox
Android runs on the Linux kernel but has no GNU user space: the C library is Bionic, and apps run on ART. Added to the kernel: Binder IPC, power management (wakelocks), mandatory SELinux.
At boot init starts the zygote process; zygote loads the framework classes and resources once and waits.
When an app launches, zygote fork()s: the loaded pages are shared via COW, which speeds up startup. The child switches to the app's unique uid, SELinux context and seccomp filter.
The app calls a service (location, camera) through Binder: the kernel moves the data into the receiver's mapped buffer with a single copy and passes along the caller's uid, which the service uses to check permissions.
Sandbox: every app is a separate Linux user, and its data is open only to itself. When memory runs low, the user-space lmkd uses PSI signals to kill unimportant background apps.
Summary
Summary: The Course's Concepts in Linux
| Topic | Concept | In Linux |
|---|---|---|
| Structures | System calls, monolithic kernel | syscall, LKM, eBPF |
| Processes | PCB, creation, IPC | task_struct, fork/exec/wait, signals, pipes |
| Threads | Kernel threading model | clone(), NPTL 1:1 |
| Scheduling | Fair sharing, real time | EEVDF, FIFO/RR/DEADLINE, sched_ext |
| Synchronization | Locks, condition variables | futex, spinlock, RCU |
| Deadlocks | Detection and prevention | lockdep, fcntl EDEADLK |
| Memory | Allocation, fragmentation | zones, buddy lists, SLUB |
| Virtual memory | Paging, replacement | 4/5-level tables, COW, MGLRU, zswap, OOM |
| File systems | i-nodes, directories, journaling | VFS, ext4, XFS, Btrfs |
| I/O | Drivers, scheduling | device files, blk-mq, io_uring |
| Multiprocessors | Load balancing, NUMA | per-CPU queues, scheduling domains |
| Security | Access control, isolation | rwx, capabilities, namespaces, SELinux, seccomp |
| Design | Separating mechanism and policy | cgroups, sched_ext: policy in user space |
Summary
Review Questions
- Which structures does
fork()copy in the kernel, and which does it share? What does COW save? - How does the shell set up
ls | wc -l? What happens if the parent does not close the pipe ends? - What is the difference between a process and a thread in Linux? Explain with clone flags.
- What changed in the fair class in Linux 6.6, and what stayed the same?
- What are the four 4-level table indices and the offset of the address
0x00007f3a1c2b5e88? - Why is SLUB better than allocating pages directly for objects like
task_struct? - With
umask 027, what permissions do a new file and a new directory get? - What is the difference between a hard link and a symbolic link? What happens when the target is deleted?
- How is consistency restored after a crash with ext4 ordered journaling?
- Container = namespaces + cgroups: what does each provide? How does it differ from a VM?
Answers
- A new task_struct, PID and kernel stack are created; page tables are copied and writable pages made read-only; open files are shared by bumping the
struct filereference count. With COW pages are copied only when written; when exec follows fork, almost nothing is copied. - pipe() → fork; child 1 does dup2(write, 1) + exec ls; child 2 does dup2(read, 0) + exec wc. If the parent keeps the write end open, wc never sees EOF and waits forever.
- There is no structural difference: both are a task_struct. A thread shares the address space, files and handlers via CLONE_VM, CLONE_FILES, CLONE_SIGHAND and CLONE_THREAD, and sits in the same TGID.
- The selection rule changed: instead of the smallest vruntime, the earliest virtual deadline among eligible tasks (v ≤ V). vruntime, the nice weights and the red-black tree stayed; a base slice replaced the latency knobs.
- PGD = 254, PUD = 232, PMD = 225, PTE = 181, offset = 0xe88.
- Same-size objects are packed side by side in a slab: no internal fragmentation, O(1) allocation and freeing, and a lock-free fast path with per-CPU lists.
- File 666 & ~027 = 0640 (rw-r-----), directory 777 & ~027 = 0750 (rwxr-x---).
- A hard link is a second name for the same i-node; the data survives deletion of the target name. A symbolic link is a separate file holding the path; it dangles when the target is deleted. A hard link cannot cross file systems.
- Data is written first to its home location and metadata to the journal, then the commit. At mount, committed transactions are replayed and uncommitted ones discarded; no full fsck is needed.
- Namespaces isolate the view (PID, network, mounts…); cgroups limit resource usage. Containers share the host kernel: light, but weaker isolation than a VM.
Summary
References
- A. S. Tanenbaum, H. Bos, Modern Operating Systems, 4th ed. (2014) / 5th ed. (2023), Pearson — Chapter 10: Case Study 1: Unix, Linux and Android.
- A. Silberschatz, P. B. Galvin, G. Gagne, Operating System Concepts, 10th ed., Wiley, 2018 — Chapter 20: The Linux System.
- R. Love, Linux Kernel Development, 3rd ed., Addison-Wesley, 2010.
- M. Kerrisk, The Linux Programming Interface, No Starch Press, 2010; man7.org manual pages.
- The Linux Kernel Documentation, docs.kernel.org — scheduler (EEVDF, sched-ext), admin-guide/mm (MGLRU, zswap), block (blk-mq), filesystems (vfs, ext4), userspace-api (seccomp, landlock).
- Android Open Source Project, source.android.com — architecture, Binder, the application sandbox.
- D. M. Ritchie, K. Thompson, “The UNIX Time-Sharing System”, Communications of the ACM, 1974.