Memory and storage
Computers use a hierarchy of memory, from tiny fast registers to huge slow storage, balancing speed, capacity, and cost.
The memory hierarchy
Memory is organised in a hierarchy: registers (fastest, smallest), cache, main memory (RAM), solid-state drives, hard disk drives, and archival storage (slowest, largest, cheapest per byte). Faster memory costs more per byte, so systems combine small amounts of fast memory with large amounts of slow storage.
Cache memory
Cache is small, very fast memory close to the CPU that stores copies of recently or frequently used data. CPUs have multiple cache levels: L1 (fastest, smallest), L2, and L3 (larger, slower, often shared between cores). A cache hit avoids a slow trip to main memory.
Locality
Caches work because programs exhibit locality. Temporal locality means recently used data is likely to be used again soon; spatial locality means data near recently used data is likely to be used soon, such as the next element of an array. Writing code that accesses memory sequentially can dramatically improve speed.
Main memory
Random-access memory (RAM), usually DRAM, holds running programs and their data. It is volatile: contents are lost when power is off. Having enough RAM matters; when it fills up, the operating system moves data to much slower storage, causing slowdowns.
Storage devices
Non-volatile storage keeps data without power. Hard disk drives (HDDs) store data magnetically on spinning platters, offering cheap capacity but slower access. Solid-state drives (SSDs) use flash memory with no moving parts, offering much faster access and greater durability, and have replaced HDDs in most laptops.
ROM and firmware
Read-only memory (ROM) and flash chips store firmware, such as the UEFI or BIOS code that starts a computer. When you power on, firmware runs tests and loads the operating system's boot loader from storage.
Memory addresses
Each byte in memory has a numerical address. A 64-bit system can in principle address 2^64 bytes, vastly more than any computer has. Older 32-bit systems were limited to 4 GB of directly addressable memory (2^32 bytes), one reason 64-bit systems became standard.
Worked example
Reading a value from L1 cache might take about 1 nanosecond, from main memory about 100 nanoseconds, from an SSD around 100 microseconds (100,000 nanoseconds), and from a hard disk around 10 milliseconds (10,000,000 nanoseconds). If a program's data fits in cache, it can run many times faster than if it constantly fetches from memory or disk. These figures are typical orders of magnitude.
Key takeaways
The memory hierarchy trades speed for capacity and cost: registers, cache, RAM, SSDs, HDDs, archives. Multi-level caches keep frequently used data close to the CPU and rely on temporal and spatial locality. RAM is volatile; running out causes slow swapping to storage. HDDs are cheap and slow; SSDs are fast and durable. ROM and flash hold firmware that boots the system. 64-bit addressing removed the 4 GB limit of 32-bit systems.
Flashcard
Registers, cache, RAM, storage
Check your understanding
Why do CPU caches improve performance?