Abstract:

This article provides a comprehensive tour of process memory management on Linux, demystifying how the operating system creates the illusion of a simple, contiguous address space for applications. It explains the fundamental concepts of virtual memory, including the distinction between virtual addresses and physical RAM frames, the role of page tables for translation, and the structure of a process's address space as a collection of Virtual Memory Areas (VMAs).

The author details the lazy allocation mechanism, where physical memory is only assigned on the "first touch" via a page fault. Key related concepts like copy-on-write (CoW) for fork() and MAP_PRIVATE mappings are explained through this lens. The article also covers memory permissions (mprotect), the performance implications of Transparent Huge Pages (THP), and the security-driven architecture of Page Table Isolation (PTI) used to mitigate the Meltdown vulnerability. Finally, it offers practical guidance on how to inspect a process's memory layout using the /proc filesystem and troubleshoot common memory-related errors.

A Guided Tour of Linux Process Memory

• Intro: The Floor Plan You Never See

  • Key Takeaway: Linux creates the illusion of a continuous, private memory space for each program, but the underlying physical RAM is fragmented. A process's address space is managed as a set of Virtual Memory Areas (VMAs), which are contiguous virtual address ranges with uniform permissions.
  • Details: The CPU uses page tables to translate a program's virtual addresses into physical memory frame addresses. Key system calls like mmap, mprotect, and munmap are used to manipulate these VMAs. Physical memory is not allocated until it is first accessed.

• mmap: Drawing a Promise

  • Key Takeaway: The mmap system call doesn't allocate physical memory. It only reserves a virtual address range and records a "promise" to provide memory when needed.
  • Details: This reserved area can be backed by a file or be "anonymous" (zero-initialized). MAP_SHARED mappings write changes back to the source file, while MAP_PRIVATE mappings create a private copy on the first write.

• The First Touch: Lazy Allocation in Action

  • Key Takeaway: When a program first tries to access memory in a valid VMA, the CPU triggers a "page fault" because no physical memory has been mapped yet. The kernel handles this fault by allocating a physical page, updating the page table, and then letting the program's instruction complete.
  • Details: A "minor fault" means the data was already in RAM (e.g., in the page cache), while a "major fault" means the kernel had to perform slow I/O from disk. Accessing an address outside any VMA or with the wrong permissions results in a SIGSEGV signal (segmentation fault).

• Copy on Write (CoW): Efficient Forking

  • Key Takeaway: When a process uses fork(), the child process doesn't get a full copy of the parent's memory. Instead, they share the same physical pages, which are temporarily marked as read-only.
  • Details: The first time either the parent or child writes to a shared page, the kernel intervenes, creates a private copy of that page for the writer, and updates its page table. This "copy-on-write" mechanism makes process creation very fast.

• Seeing What's Mapped: The /proc Filesystem

  • Key Takeaway: You can inspect a process's memory layout and usage through the virtual /proc filesystem.
  • Details:
    • /proc/<pid>/maps: Shows the address ranges, permissions, and backing files for all VMAs.
    • /proc/<pid>/smaps: Provides detailed memory accounting per VMA, including Resident Set Size (RSS), private vs. shared pages, and huge page usage.
    • /proc/<pid>/pagemap: An advanced tool (requiring privileges) that provides per-page information, including whether a page is in RAM and its physical address (Page Frame Number).

• Transparent Huge Pages (THP): Gaining Performance

  • Key Takeaway: To improve performance and reduce the overhead of memory translation, Linux can automatically use large 2 MiB "huge pages" instead of standard 4 KiB pages for certain memory regions.
  • Details: THP is primarily used for anonymous memory. A background kernel thread, khugepaged, can merge smaller pages into huge pages. Usage can be monitored via the AnonHugePages field in /proc/<pid>/smaps.

• Meltdown: A Tale of Two Page Tables

  • Key Takeaway: To defend against the Meltdown CPU vulnerability, Linux implements Page Table Isolation (PTI). It maintains two separate page tables: a minimal one for when the process is in user mode and a complete one for when it's in kernel mode.
  • Details: This prevents speculative execution attacks from leaking kernel memory to user processes but introduces a small performance cost due to the constant switching of page tables on every system call and interrupt.

• Troubleshooting Common Issues

  • Key Takeaway: The article concludes with a practical guide for diagnosing common memory-related errors.
  • Details:
    • mmap returning EINVAL: Often caused by a file offset that is not page-aligned.
    • Receiving SIGBUS: Typically happens when accessing a file mapping beyond the actual end of the file.
    • Memory usage balloons after fork(): This is the expected result of copy-on-write as the processes modify their shared memory.