Chapter 2: Operating-System Structures
Introduction
Learning Outcomes
Explain the services an OS provides and the concepts of process, address space, file, mounting and pipe.
Trace step by step the switch from user to kernel mode, parameter passing and the basic POSIX calls.
Compare monolithic, layered, microkernel, modular, hybrid and exokernel structures, with their strengths and weaknesses.
Describe virtual machines and containers, the linker/loader and the boot steps.
Introduction
Why Does Structure Matter?
- An OS is made of many components: process, memory, file, I/O and network management, and more.
- The strategy for where these components run (kernel / user mode) and how they communicate (function call / message) is called the OS structure.
- Structure directly affects how we manage complexity, make changes, debug and port.
Services
Operating-System Services
User interfaces and programs reach the services through system calls (adapted from Silberschatz, Figure 2.1).
Services
Whom Do the Services Serve?
- Interface: command line, GUI, touch, voice.
- Program execution: load, run, terminate normally or abnormally.
- I/O operations: a program reaches a device through the OS, not directly.
- File system: create, delete, read, write, search, permissions.
- Communication: shared memory or message passing (local / network).
- Error detection: hardware, device and program errors.
- Resource allocation: CPU, memory and storage are shared among many processes.
- Accounting: who used how much of what; logs.
- Protection: processes cannot harm each other or the kernel.
- Security: authentication, authorization, defense against outside attacks.
ls, cp, ps, package managers, compilers, file managers…Basic Concepts
Process
- A process is a program in execution; the program is a passive file on disk, the process its active instance.
- Each process has an address space (code, data, stack), registers (PC, SP), open files, alarms, related processes and other resources.
- The OS keeps this information in the process table; a suspended process resumes where it left off thanks to its PCB.
- Processes can talk to each other through IPC (interprocess communication).
Basic Concepts
Process Tree
A process can create child processes with fork(); the children create their own children. The result is a tree.
At first only process A exists. On Linux the root of the whole tree is systemd (or init), PID 1.
Process A calls fork() twice: B and C are A's children (A is their parent).
Process B creates three children: D, E and F. The pstree -p command shows this tree.
Basic Concepts
Address Space and Memory Segments
- Address space: the set of addresses a process can use (0 … max). On x86-64 Linux user space is 128 TiB.
- Many processes are in memory at once; the OS protects them from each other and from the kernel.
- Text: machine code; read-only, shared by processes running the same program and by all threads.
- Data + BSS: global and static variables. Heap: memory allocated dynamically at run time. Both are shared by the process's threads.
- Stack: parameters, local variables, return addresses; each thread has its own stack.
- Virtual memory allows address spaces larger than physical memory.
Basic Concepts
Files and the Directory Hierarchy
- File: a named unit that stores data persistently (text, images, video, programs…).
- The OS hides disk blocks and device details and offers a clean abstraction.
- Files live in a directory tree. A path name can be absolute (
/home/ali/notes.txt) or relative to the working directory (ali/notes.txt). - An opened file gets a small integer, the file descriptor (fd); 0, 1, 2 = stdin, stdout, stderr.
- Special files: devices look like files. Block devices (
/dev/sda,/dev/nvme0n1) and character devices (/dev/tty, serial port).
Basic Concepts
Mounting a File System
Before mounting, the file system on the USB stick is a separate tree; no path name reaches it.
mount attaches the root of the new file system to a mount point in the existing tree (the empty directory /mnt).
Once mounted, the USB files are part of the hierarchy. Windows usually uses drive letters (C:, D:) instead.
Basic Concepts
The mount Command and Its Options
$ lsblk # list disks and partitions $ sudo mount /dev/sdb1 /mnt # mount sdb1 on /mnt $ sudo mount -o ro /dev/sdb1 /mnt # mount read-only $ sudo mount -o remount,rw /mnt # remount read-write $ findmnt /mnt # show mount info $ sudo umount /mnt # unmount
ro / rwMount read-only or read-write. ro is the safe choice for recovery and forensics.
/etc/fstabFile systems to mount automatically at boot are listed here (or in systemd mount units).
A plugged-in USB stick is mounted automatically under /run/media/… by udisks2.
Basic Concepts
Pipe
- A one-way byte channel buffered in the kernel that connects two processes; they use it like a file, with
read/write. - Anonymous: created with
pipe(), between parent and child. Named (FIFO): created withmkfifo, has a name in the file system, usable by unrelated processes.
$ ls -l | grep "notes"
The shell first calls pipe(fd): the kernel opens a buffer with two ends (fd[0] read, fd[1] write).
The shell forks twice; with dup2, ls's stdout goes to the write end and grep's stdin comes from the read end.
ls writes its listing into the pipe; grep reads it and prints only the lines containing “notes”.
Flow control is in the kernel: the writer blocks when the buffer is full, the reader when it is empty. When ls finishes and the write end closes, grep sees end of file (EOF).
Basic Concepts
Deadlock
- If every process in a set waits for a resource held by another process in the set, none can proceed: deadlock.
- Like cars entering an intersection from four directions and waiting for each other.
Arrow from box to process: the resource is held by that process. Arrow from process to box: the process requests it (dashed = will request later).
Protection
Protection with Dual Mode
- The OS protects itself and processes through the hardware's user / kernel mode split: I/O, setting the timer, disabling interrupts and changing page tables are possible only in kernel mode.
- If a privileged instruction is attempted in user mode, the hardware traps and the OS kills the process. The kernel is entered only through system calls, interrupts and exceptions.
- x86 has rings 0–3, but Linux and Windows use only 0 and 3; ARM has EL0–EL3, with EL2 reserved for virtualization.
- Mode separation alone is not enough: a timer is needed to take the CPU back, and an MMU to keep process memories apart.
Protection
Timer
- Goal: stop a process from looping forever and never giving up the CPU.
- The OS loads a value into a counter; this is a privileged instruction.
- The counter decrements on every tick of the physical clock; at zero an interrupt fires and the OS regains control.
- The OS preempts a process that has used up its time slice and gives the CPU to another (preemptive scheduling). The process is killed only if a CPU-time limit (e.g.
RLIMIT_CPU) is exceeded. - Linux's periodic tick is 100–1000 Hz; tickless (NO_HZ) mode, which stops ticks when idle, saves energy.
Before running A, the OS loads 3 into the counter.
Each clock tick decrements the counter: 2.
Counter 1; A is still on the CPU.
Counter 0: timer interrupt. The CPU switches to kernel mode and the interrupt handler runs.
A's slice is used up, so the scheduler picks B and reloads the counter.
System Calls
What Is a System Call?
- The defined interface programs use to reach OS services (files, memory, processes, time, network…).
- Requested by the application, executed in kernel mode by the kernel.
- Programmers usually reach calls through an API, not directly: POSIX (glibc, musl), the Windows API, the Java API. The API gives portability and convenience.
- The set of calls varies by system: Linux on x86-64 has about 370. The core ideas are shared.
strace(Linux),dtruss(macOS) and Process Monitor (Windows) show the calls a program makes.
System Calls
API → System-Call Interface → Kernel
The application calls the open() wrapper in libc; the wrapper puts the call number in a register and executes the trap instruction.
The kernel's system-call interface uses the number as an index into the system-call table (x86-64 Linux: open = 2).
The pointer in the table leads to the kernel implementation: the path is resolved, permissions are checked, an open-file entry is created.
The result (the lowest free descriptor: 3, since 0, 1, 2 are taken) returns to the user. Note: current glibc actually implements open() with openat (257).
System Calls · Step through the code
Code Trace: The System-Call Table
typedef long (*syscall_t)(long, long, long); syscall_t table[3] = {sys_read, sys_write, sys_open}; long syscall_entry(long no, long a, long b, long c) { if (no < 0 || no >= 3) return -ENOSYS; /* bad number */ return table[no](a, b, c); }
The libc wrapper puts the call number in rax and executes syscall. The hardware enters kernel mode and jumps to the single entry point: no = 1 (write on Linux x86-64).
0 ≤ 1 < 3: the number is valid.
table[1] is a function pointer: sys_write is called and returns 3, the number of bytes written.
A new call: open("notes.txt", 0), no = 2.
0 ≤ 2 < 3: the number is valid.
table[2] is a function pointer: sys_open is called and returns 3, the new descriptor.
A new call: syscall(99), no = 99.
The number is outside the table. Without the check, table[99] would jump to a random kernel address. The call is rejected with −ENOSYS (−38).
libc turns the negative value into errno = ENOSYS and returns −1 to the program. The kernel never uses a value from user space without validating it.
System Calls
Example: Copying One File to Another
Even getting a name from the user takes two calls: write a prompt to the screen (write), read from the keyboard (read).
The same is repeated for the output file.
If open fails (no such file, no permission) it returns −1; the program prints an error and exits.
open(…, O_WRONLY | O_CREAT | O_EXCL): fails if the file already exists.
The real work: a read/write loop until read returns 0 (end of file).
Every open file is closed with close.
The user is informed.
The process ends with exit. Even a simple program like this makes dozens of calls; count them with strace -c cp a b.
System Calls · Step through the code
Code Trace: Copying with read/write
char buf[4096]; int in = open("src", O_RDONLY); int out = creat("dst", 0644); ssize_t n; long total = 0; while ((n = read(in, buf, 4096)) > 0) { write(out, buf, n); total += n; } close(in); close(out); printf("%ld bytes\n", total);
“src” is a 10,000-byte file. open returns the lowest free descriptor: 0, 1, 2 are taken, so in = 3; offset 0.
An empty destination file is created: out = 4.
read enters the kernel and asks for at most 4096 bytes: n = 4096. The kernel advances the file offset by 4096.
The n bytes in the buffer (as many as were read, not 4096) are written to the destination.
total = 4096. Each iteration costs two system calls: one read, one write.
read again: n = 4096, offset 8192.
4096 bytes are written, total = 8192.
Only 10,000 − 8192 = 1808 bytes are left: read returns fewer than requested, n = 1808. This is not an error.
1808 bytes are written, total = 10000.
The offset is at end of file: read returns 0 (EOF). The condition is false and the loop ends.
Both descriptors are closed; 3 and 4 can be reused.
Output: 10000 bytes. 12 system calls in all (printf also does a write). With a 1-byte buffer, reading and writing alone would take about 20,000 calls.
System Calls
The 11 Steps of read(fd, buffer, nbytes)
1–3: Parameters are pushed on the stack (the historical 32-bit x86 convention in Tanenbaum's example). On x86-64 the first six parameters go in registers.
4: The library's read() is called like an ordinary function; the return address goes on the stack.
5: The library puts the call number in the register the kernel expects (Linux x86-64: rax = 0).
6: The trap instruction (x86-64: syscall) runs; the CPU enters kernel mode and jumps to a fixed kernel address.
7: The kernel looks at the call number and dispatches to the right handler through the system-call table.
8: The handler copies up to nbytes bytes from fd into buffer. If no data is ready, the process blocks and the CPU goes to another process.
9: When done, control returns to user mode at the instruction after the trap; the result (bytes read) is in a register.
10: The library's read() returns to the user program.
11: The program pops the parameters off the stack (SP is incremented) and continues with the next instruction.
System Calls
Passing Parameters to System Calls
- Registers: the fastest way; Linux passes up to 6 parameters like this.
- Block / table in memory: parameters in memory, their address in a register (figure). E.g.
clone3(struct clone_args *). - Stack: the program pushes the parameters, the OS pops them.
| x86-64 Linux | Role |
|---|---|
rax | call number; result on return |
rdi, rsi, rdx | 1st, 2nd, 3rd parameter |
r10, r8, r9 | 4th, 5th, 6th parameter (r10 instead of rcx from the C convention) |
syscall | enter kernel; rcx and r11 clobbered |
| ARM64 | x8 = number, x0–x5 parameters, svc #0 |
errno and returns −1 to the caller.System Calls
Code Trace: Writing to the Screen with write()
#include <unistd.h> // write() #include <string.h> // strlen() int main(void) { const char *msg = "Hello, Türkiye!\n"; size_t len = strlen(msg); // 17 bytes ssize_t n = write(1, msg, len); // 1: stdout if (n == -1) return 1; // error return 0; }
msg points to the string in the read-only data segment.
“Hello, ” 7 + “Türkiye” 8 (ü takes 2 bytes in UTF-8) + “!” and “\n” 2 = 17 bytes. A hard-coded 15 would cut the output to “Hello, Türkiye”.
The libc wrapper: rax=1 (write), rdi=1, rsi=msg, rdx=17; then syscall.
In kernel mode, sys_write passes the 17 bytes to the terminal behind fd 1.
Back in user mode: n = 17, the number of bytes written. It could be smaller (a partial write).
n is not −1, so no error. On error it would be −1 and errno would be set.
main returns 0; libc ends the process with exit_group(0).
System Calls
Code Trace: read() and Blocking
#include <unistd.h> #include <stdio.h> int main(void) { char buf[100]; ssize_t n = read(0, buf, sizeof(buf)); if (n == -1) { perror("read error"); return 1; } write(1, buf, n); // exactly n bytes return 0; }
buf is allocated on the stack; its contents are uninitialized.
fd 0 = stdin (keyboard). No line yet: the process blocks and the scheduler runs another process.
The user types “hello” and presses Enter; the terminal driver echoes the characters and buffers the line.
Once the line is complete, 6 bytes are copied into buf, the process wakes up and read returns 6. No '\0' is appended.
n = 6, no error; perror is skipped.
write outputs exactly 6 bytes. Since buf is not a C string, printf("%s", buf) would be wrong.
The program exits with 0.
System Calls
The Standard C Library and System Calls
The program calls printf; it is not a system call but a library function running in user mode.
libc formats the text and puts it in the stdout buffer. On a terminal the buffer is line-buffered; redirected to a file or pipe it is fully buffered.
Because “\n” arrived, libc makes the system call write(1, buffer, 8); the kernel passes the bytes to the terminal.
write returns 8 and printf returns to its caller. strace ./welcome shows only write, not printf.
Headers: stdio.h (printf, fopen), stdlib.h (malloc, exit), string.h (strlen, memcpy), math.h, time.h; POSIX unistd.h (fork, read, write). Implementations: glibc, musl, UCRT on Windows. Functions like strlen live entirely in the library; ones like write wrap a system call.
System Calls
Types of System Calls
create, terminate, load, wait, signal
fork, execve, exit, waitpid, killcreate, delete, open, close, read, write, attributes
open, read, write, close, statrequest / release device, read, write, control
ioctl, read, write, mmaptime, date, system and process information
getpid, uname, clock_gettimeconnect, send/receive messages, shared memory
pipe, socket, sendmsg, shm_openget / set permissions, user identity
chmod, chown, umask, setuidWindows equivalents: CreateProcess, ReadFile, DeviceIoControl, GetSystemTime, CreatePipe, SetFileSecurity… (Silberschatz, Figure 2.8)
System Calls
POSIX Calls: Process Management
| Call | Description |
|---|---|
pid = fork() | Creates a child process that is a copy of the parent. |
pid = waitpid(pid, &statloc, options) | Waits for a child to terminate; writes its exit status to statloc. |
s = execve(name, argv, envp) | Replaces the process's memory image with a new program; does not return on success. |
exit(status) | Terminates the process and passes the status to the parent. |
errno. pid: process ID, s: return code. On Windows a single CreateProcess call does the work of fork + execve.System Calls
Code Trace: A Simple Shell
#define TRUE 1 while (TRUE) { /* forever */ type_prompt(); /* show prompt */ read_command(command, parameters); if (fork() != 0) { /* parent */ waitpid(-1, &status, 0); } else { /* child */ execve(command, parameters, 0); perror("execve"); /* on failure */ _exit(127); } }
The shell prints a prompt and waits for the user.
The user types ls; the command and its parameters are parsed.
fork makes a copy of the shell. In the parent the result is 1045 (≠ 0); in the child it is 0.
The parent waits for any child to finish with waitpid(-1, …); it blocks.
The child replaces itself with the ls program via execve (the PID stays the same); ls prints its output. Real shells search PATH for the command.
ls finishes with exit(0); the parent's waitpid returns and status holds the exit status.
The loop starts over and a new prompt is printed.
System Calls
POSIX Calls: File Management
| Call | Description |
|---|---|
fd = open(file, how, …) | Opens a file for reading, writing or both (creating it if needed). |
s = close(fd) | Closes an open file. |
n = read(fd, buffer, nbytes) | Reads up to nbytes bytes from the file into the buffer. |
n = write(fd, buffer, nbytes) | Writes data from the buffer to the file. |
position = lseek(fd, offset, whence) | Moves the file position pointer (SEEK_SET, SEEK_CUR, SEEK_END). |
s = stat(name, &buf) | Gets the file's status information (size, permissions, times). |
fd: file descriptor, n: byte count, position: offset from the start of the file. Current Linux also offers openat, pread/pwrite, statx.
System Calls
POSIX Calls: Directory Management and Others
| Call | Description |
|---|---|
s = mkdir(name, mode) | Creates a new directory. |
s = rmdir(name) | Removes an empty directory. |
s = link(name1, name2) | Creates a new directory entry name2 pointing to the file name1 refers to. |
s = unlink(name) | Removes a directory entry. |
s = mount(special, name, flag) | Mounts a file system (Linux: mount(src, target, type, flags, data)). |
s = umount(special) | Unmounts a file system. |
s = chdir(dirname) | Changes the working directory. |
s = chmod(name, mode) | Changes a file's protection (permission) bits. |
s = kill(pid, signal) | Sends a signal to a process (not only to kill it). |
seconds = time(&seconds) | Seconds elapsed since 1 January 1970 00:00 UTC (the Unix epoch). |
A 32-bit time_t overflows on 19 January 2038 (the year-2038 problem); current systems use a 64-bit time_t.
System Calls
link and unlink: One File, Two Names
In UNIX a directory is a table of (i-node number, name) pairs. The file's real information lives in the i-node (inode).
Start: /usr/jim/memo refers to i-node 70; link count 1.
link("/usr/jim/memo", "/usr/ast/note"): a new entry referring to the same i-node (70) is added to /usr/ast. No data is copied; the count becomes 2.
unlink("/usr/jim/memo") removes only the entry; the file lives on as note. When the count reaches 0 (and no one holds it open) its blocks are freed.
System Calls · Step through the code
Code Trace: Link Count and an Open File
int fd = open("/usr/jim/memo", O_RDONLY); link("/usr/jim/memo", "/usr/ast/note"); unlink("/usr/jim/memo"); unlink("/usr/ast/note"); n = read(fd, buf, 100); /* still readable */ close(fd);
A directory is just (i-node number, name) pairs. memo refers to i-node 70; link count 1, file size 85 bytes.
open resolves the path and opens i-node 70: fd = 3, the i-node's open count is 1.
link adds a new entry in /usr/ast referring to the same i-node. No data is copied; links = 2.
unlink removes only the directory entry; the file lives on as note. Links = 1.
The last name is gone too: links = 0. The file has no path any more, but since fd 3 holds it open, the i-node and blocks are not deleted.
read goes straight to the i-node through the descriptor; no name is needed: n = 85.
The last descriptor is closed: links = 0 and open = 0. Only now does the kernel free the i-node and data blocks. Temporary files are deleted while open this way.
System Calls
Win32 API Equivalents of UNIX Calls
| UNIX | Win32 |
|---|---|
fork | CreateProcess |
waitpid | WaitForSingleObject |
execve | (none: CreateProcess) |
exit | ExitProcess |
open | CreateFile |
close | CloseHandle |
read | ReadFile |
write | WriteFile |
lseek | SetFilePointerEx |
stat | GetFileAttributesEx |
| UNIX | Win32 |
|---|---|
mkdir | CreateDirectory |
rmdir | RemoveDirectory |
link | CreateHardLink |
unlink | DeleteFile |
mount | SetVolumeMountPoint |
umount | DeleteVolumeMountPoint |
chdir | SetCurrentDirectory |
chmod | (none: ACLs, SetFileSecurity) |
kill | (none: TerminateProcess) |
time | GetSystemTime |
Nt… functions in ntdll.dll (e.g. NtCreateFile). Their numbers change between versions and are undocumented, so programs use the API. The link and mount equivalents listed as “none” in Tanenbaum 3e exist today.Structures
Operating-System Structures: Overview
Not a placement but an organizing principle: components are split into hierarchical layers (THE, MULTICS rings).
A separate axis: the hardware itself is virtualized and several OSes run on top of it.
Structures
Simple Structure: MS-DOS
- No clear structure: it was written to be small and simple, for limited hardware.
- Interfaces and levels of functionality are not well separated; applications can reach BIOS routines and the hardware directly.
- The Intel 8088 had no dual mode, so there was no hardware protection: one faulty program could crash the whole system.
- No clear boundaries between modules and no data hiding.
- Silberschatz treats this as a “simple structure”, separate from monolithic kernels.
Structures
Monolithic Systems
The whole OS is linked as a single program running in kernel mode. Any procedure can call any other directly.
- Main program: looks at the call number and invokes the requested service procedure.
- Service procedures: carry out the system calls.
- Utility procedures: work shared by several services (e.g. fetching data from user memory).
Most monolithic kernels today also accept drivers and file systems as loadable modules at run time.
Structures
Traditional UNIX and Linux: The Monolithic Kernel
Strengths
- Direct function calls: no IPC, no mode switches, fast.
- Relatively simple, integrated design and implementation.
Weaknesses
- One driver bug can crash the whole kernel.
- A huge code base (Linux passed 40 million lines in 2025): debugging and maintenance are hard.
Structures
Layered Approach
Layer 0: switches between processes on interrupts and timer expiry; layers above need not think about the single CPU.
Layer 1: allocates memory to processes and moves pages between the drum (the disk of the day) and memory.
Layer 2: handles communication between each process and the operator console.
Layer 3: manages I/O devices; layers above work with abstract devices.
Layer 4: user programs; they need not deal with memory, console or I/O details.
Layer 5: the system operator. Dijkstra's THE system (1968) had these six layers.
The OS is divided into layers: the bottom layer is closest to the hardware, the top to the user. Each layer uses the services of only the layers below it.
Pros
- Debugging and verification proceed layer by layer; a bug is sought in the relevant layer.
- Abstraction and data hiding come easily.
Cons
- Defining the layers properly is hard (dependencies often run both ways).
- Each layer crossing adds overhead. The idea lives on today in network stacks.
Structures
Microkernels
- Everything nonessential is moved out of the kernel; only IPC (message passing), basic scheduling, address-space management and interrupt handling remain.
- Drivers, file systems and the network stack run as user-mode server processes; new services are added in user space without changing the kernel.
- Examples: Mach, MINIX 3, QNX Neutrino (automotive), the L4 family, the formally verified seL4, and the Zircon kernel of Google Fuchsia. macOS, by contrast, uses XNU, a hybrid derived from Mach.
- Mechanism in the kernel, policy outside: the kernel can run processes by priority; a user-mode component decides how priorities are assigned.
Pros
- A crashed driver does not bring the system down; it can be restarted.
- Small kernel: easy to test, verify and port.
- More secure: a small attack surface.
Cons
- Messages and mode switches add overhead; L4 designs greatly reduced it.
Structures
Reading a File on a Microkernel
The application sends a message to the file-system server; the message passes through the kernel's IPC mechanism.
The kernel delivers the message to the file-system server in user mode.
The file system works out which block to read and sends a message, again through the kernel, to the disk driver.
The driver reaches the hardware through I/O the kernel permits; replies come back the same way. In a monolithic kernel this is a single chain of function calls.
Structures · Step through the code
Code Trace: send/receive on a Microkernel
/* client */ m.type = READ; m.file = 7; m.offset = 0; send(FS, &m); receive(FS, &m); /* wait for reply */ /* file-system server (FS) */ while (1) { receive(ANY, &m); block = find_block(m.file, m.offset); send(DISK, &block); receive(DISK, &data); m.result = data; send(m.source, &m); }
FS and the disk driver are each a user-mode process; both wait for work in receive. The client writes its request into a message structure.
send traps into the kernel: the kernel copies the message into FS's buffer and wakes FS. The client blocks in receive for the reply.
FS's receive(ANY) returns; the sender is recorded in m.source. FS finds that offset 0 of file 7 is in block 120.
FS cannot touch the disk directly either: it sends a second message to the driver and waits for the reply.
The driver reads the block through I/O the kernel permits and returns the data to FS in a third message. FS prepares the reply.
The fourth message carries the reply to the client; the client becomes ready.
FS goes back to the top of its loop and waits for new work.
The client's receive returns: the data is in m.result. One read took 4 messages, each with a user ↔ kernel switch; a monolithic kernel would finish the same job with function calls inside one system call.
Structures
MINIX 3: A Microkernel Example
Monitors the drivers' health and restarts a crashed driver (e.g. TTY) without the user noticing: a self-healing system.
Drivers are in user mode; they reach I/O ports only through kernel calls, to the extent allowed. A faulty driver cannot corrupt memory.
Structures
The Client–Server Model
On a microkernel, processes fall into two classes: clients that request services and servers that provide them. They communicate only by messages.
A client need not know whether a message is handled locally or on another machine on the network: the model extends naturally to distributed systems (today's web and microservice architectures).
Message passing is not secure by itself: security depends on the kernel checking which process may send messages to which server.
Structures
The Tanenbaum–Torvalds Debate (1992)
- In his post “LINUX is obsolete” he called the monolithic design an outdated 1970s approach.
- A microkernel is more modular, flexible and portable; damage from a bug in one component is contained.
- As hardware gets faster, the overhead of message passing becomes negligible.
- A monolithic kernel is simpler to design and implement; direct function calls are faster.
- No message passing between processes is needed; in practice, code that works is what counts.
- The open-source development model let Linux evolve quickly.
Structures
Modular Structure: Loadable Kernel Modules
- The kernel holds only core components; other services are loaded at boot or at run time as modules (Linux LKM, FreeBSD, Solaris).
- Like layering (defined interfaces) but more flexible: any module can call any other. Like a microkernel, but no messages are needed: modules run in kernel mode.
At boot the kernel and the required modules from initramfs are loaded. lsmod lists the loaded modules.
A USB stick is plugged in: a udev event loads the driver (with its dependencies) via modprobe usb-storage; the vfat module is loaded when the stick is mounted.
The module registers itself with the kernel and works through direct function calls: fast, but no isolation; a faulty module can crash the kernel.
When its use count is 0, modprobe -r (or rmmod) unloads the module. Modules must match the kernel version.
Structures
Hybrid Kernels: Windows and macOS
Windows NT: microkernel ideas (subsystems, object/message-based design) are kept, but for performance the executive, graphics and drivers were moved into kernel mode.
macOS / iOS XNU: the Mach microkernel and the BSD layer share one kernel address space: microkernel modularity + monolithic speed.
Structures
Exokernels
- A research architecture by Engler and Kaashoek at MIT (1995): the kernel Aegis and the library OS ExOS. Not a commercial OS.
- A VM gets a copy of the machine; an exokernel hands out a share of a resource and checks ownership, with no remapping layer needed.
- Separates management from protection: the kernel only tracks who owns which resources; abstractions (files, virtual memory) come from application-level LibOSes.
- High performance through application-specific memory management and caching, but applications are complex to build. The idea lives on in unikernels and kernel-bypass I/O such as DPDK / SPDK.
Structures
Comparing the Structures
| Structure | What is in the kernel? | Strength | Weakness | Example |
|---|---|---|---|---|
| Simple | everything, unrestricted | small, simple | no protection | MS-DOS |
| Monolithic | all services | speed | no fault isolation | UNIX, Linux |
| Layered | hierarchical layers | easy to verify | overhead, hard to define | THE, MULTICS |
| Microkernel | IPC, scheduling, memory | reliability | message overhead | MINIX 3, QNX, seL4 |
| Modular | kernel + modules | flexibility, speed | module = full privilege | Linux, FreeBSD |
| Hybrid | micro + selected services | balance | complexity | Windows, macOS |
| Exokernel | protection only | app-specific speed | hard to develop | Aegis/ExOS |
Real systems are not pure: Linux is monolithic + modular, Windows and macOS are hybrid, and nearly all are layered to some degree.
Virtual Machines
The Virtual-Machine Concept
- A hypervisor (VMM) creates virtual machines on one physical machine, each looking like an independent computer.
- Each guest OS believes it has its own CPU, RAM, disk and I/O devices; different OSes can run concurrently on the same hardware.
- The idea goes back to IBM VM/370 (1972); today it underpins cloud computing: server consolidation, isolation, snapshots, live migration.
- A hypervisor runs on the hardware (type 1) or on a host OS (type 2). A container carries no kernel of its own; it isolates processes that share one kernel.
Hardware and Structure
Symmetric Multiprocessing and Multicore Architecture
SMP: each CPU has its own registers and cache and shares memory; every CPU can do any work, the OS included. Kernel data structures must be protected by locks (Linux's “big kernel lock” was removed entirely in 2011).
Multicore chip: cores on the same chip share nearby caches and communicate faster with less energy. P/E (performance/efficiency) cores, common in 2026, give the scheduler new work.
Linking and Booting
Compiling, Linking and Loading
The compiler turns source code into a relocatable object file: addresses are not final yet and external symbols (e.g. printf) are unresolved.
The linker combines object files and static libraries, resolves symbols, fixes up addresses and produces an executable.
The shell calls execve; the kernel's loader reads the file header, maps the segments into the address space and starts the dynamic linker (ld-linux.so).
Shared libraries (libc.so.6) are linked at run time and shared among processes. Formats are OS-specific: ELF (Linux), PE (Windows), Mach-O (macOS); so programs cannot move directly between OSes.
Linking and Booting
Booting the Operating System
Booting is everything from loading the kernel into memory to starting the first user process. The old BIOS (1981, 16-bit real mode) ran the MBR code in the disk's first sector; UEFI loads a file (.efi), not a sector. On Linux systemd-analyze shows the time of each stage.
At power-on the CPU resets and starts executing firmware in flash from a fixed address set by the hardware.
UEFI tests memory and devices with the power-on self-test (POST), then, following the order in NVRAM (NVMe, USB, network/PXE), loads the boot loader from the EFI system partition (FAT32) on a GPT disk. With Secure Boot on, its signature is verified.
The boot loader shows a menu, puts the chosen kernel and the initial RAM disk (initramfs) in memory and jumps to the kernel.
The kernel starts memory management, the scheduler and drivers; scripts in initramfs load the modules needed.
The real root file system is mounted and initramfs is discarded.
The kernel starts PID 1 (systemd), which brings up services, the network and the login screen. From then on every process descends from PID 1. On Windows: winload.efi → ntoskrnl.exe → smss.exe → winlogon.exe.
Summary
Review Questions
- Why can't a user-mode program execute I/O instructions directly? What happens if it tries?
- What is the difference between
printfandwrite? Which one is a system call? - Why is
fork()“called once, returns twice”? What does it return in the parent and in the child? - What happens if a driver crashes on a microkernel? On a monolithic kernel?
- Do loadable modules make Linux a microkernel? Why?
- What is the first user process the kernel starts at boot? Why does it matter?
Show answers
- I/O is privileged; with the mode bit at 1 the hardware traps and the OS kills the process (protection).
printfis a library function that formats in user mode; when needed it makes thewritesystem call.- After the call there are two processes, both continuing from the same point: the parent gets the child's PID, the child gets 0 (error: −1).
- On a microkernel only that server process crashes, and it can be restarted; on a monolithic kernel the whole kernel can go down.
- No: modules run in kernel mode in the same address space and communicate by function calls, not messages.
- PID 1 (
systemdorinit): it starts the services and the login screen; every other process descends from it, and it adopts processes whose parent has died.
Summary
References
- Silberschatz, Galvin and Gagne, Operating System Concepts, 10th ed., Wiley, 2018 (Chapters 2 and 18).
- Tanenbaum and Bos, Modern Operating Systems, 4th ed., Pearson, 2014 (Sections 1.5–1.7 and Chapter 7).
- Arpaci-Dusseau and Arpaci-Dusseau, Operating Systems: Three Easy Pieces (OSTEP), version 1.10, pages.cs.wisc.edu/~remzi/OSTEP.
- Linux man pages:
syscalls(2),syscall(2),fork(2),pipe(7),mount(8). - Engler, Kaashoek and O’Toole, “Exokernel: An Operating System Architecture for Application-Level Resource Management”, SOSP 1995.