Memory, threads, and the kernel API
Programs see memory through the kernel's virtual memory system, share it among threads, and rely on kernel interfaces for everything from timers to containers.
Virtual memory layout
Each process has its own virtual address space. A typical layout contains the program's code (text), initialised and uninitialised data, a heap that grows upwards, memory-mapped regions for shared libraries and files, and a stack that grows downwards, with the kernel's memory protected above. /proc/PID/maps shows a process's mappings and permissions.
Allocating memory
malloc() and free(), from the C library, manage heap memory. For small requests, malloc carves space from regions obtained with brk() or mmap(); large requests often get their own mmap() mapping. The kernel allocates physical pages lazily, only when a page is first touched, so programs can reserve far more virtual memory than they use. Memory overcommit means allocations can succeed even when physical memory is short.
Page faults and protection
When a program touches a virtual page not yet in memory, the CPU raises a page fault and the kernel supplies a page, by allocating one, reading from a file, or bringing it back from swap. Pages carry read, write, and execute permissions; accessing memory in a forbidden way, such as writing to read-only code or dereferencing a null pointer, triggers SIGSEGV.
Shared memory and IPC
Processes can communicate through pipes, Unix domain sockets (fast local sockets used by many services), message queues, signals, and shared memory created with shm_open() and mmap(), which lets processes read and write the same pages at memory speed, with semaphores or mutexes for coordination. D-Bus provides a higher-level message bus used by desktops and systemd.
POSIX threads
The pthreads API creates threads with pthread_create(), waits with pthread_join(), and synchronises with mutexes (pthread_mutex_lock and unlock), condition variables, and read-write locks. Threads share the heap and global variables, so unsynchronised access causes data races. Linux implements futexes (fast user-space mutexes), which avoid system calls when locks are uncontended.
Time and timers
clock_gettime() reads clocks: CLOCK_REALTIME for wall-clock time, which can jump when adjusted, and CLOCK_MONOTONIC for measuring intervals, which never goes backwards. nanosleep() sleeps; timerfd and POSIX timers deliver timer events. Linux time is commonly measured from the Unix epoch, 00:00:00 UTC on 1 January 1970, and 64-bit time values avoid the year-2038 overflow of 32-bit seconds.
Namespaces and containers
Namespaces give processes isolated views of system resources: PID namespaces (their own process IDs), mount namespaces (their own file-system tree), network namespaces (their own interfaces and routes), user namespaces (mapped user IDs), and UTS, IPC, and cgroup namespaces. Combined with cgroups for resource limits and seccomp for filtering system calls, they form the basis of containers such as Docker.
Worked example
A C program calls malloc(1 GiB) on a machine with 8 GiB of RAM, then writes to only the first megabyte. top shows the process with a virtual size (VIRT) of over 1 GiB but a resident size (RES) of only about 1 MiB, because the kernel allocated physical pages only for the pages actually touched. If the program then wrote to all 1 GiB, page faults would allocate about 262,144 pages of 4 KiB each.
Key takeaways
Each process has a virtual address space with code, data, heap, mappings, and stack, visible in /proc/PID/maps. malloc uses brk and mmap; physical pages are allocated lazily on first touch, and overcommit lets allocations exceed RAM. Page faults bring in pages; permission violations raise SIGSEGV. IPC options include pipes, Unix domain sockets, message queues, shared memory with mmap, and D-Bus. pthreads provide threads, mutexes, and condition variables, built on futexes. CLOCK_MONOTONIC measures intervals; time counts from the 1970 epoch, with 64-bit values avoiding 2038 overflow. Namespaces, cgroups, and seccomp underpin containers.
Flashcard
Process's private view of memory
Check your understanding
Why does top show a process with 2 GiB VIRT but only 50 MiB RES?