Abstract:

This post details the successful creation of a bare-metal, user-mode GPU debugger proof-of-concept (PoC) for AMD RDNA3 (Navi 31) architecture, bypassing standard APIs like Vulkan and ROCm. The approach involves low-level interaction with the Kernel Mode Driver (KMD) via libdrm for command submission (PM4 packets) and exploiting privileged hardware debugging features. A critical component is the utilization of the undocumented TBA/TMA (Trap Handler Base/Memory Address) registers, accessed via the privileged AMDGPU Debugfs interface, to force a trap/pause state. The author developed a custom assembly trap handler to save wave context (registers) into CPU-accessible memory, enabling instruction-level stepping. As far as I can tell, the reliance on privileged kernel interfaces (debugfs) and an unstable context-saving workaround (limited to single wave execution) makes this PoC inherently fragile, but it convincingly demonstrates the technical possibility of deep GPU debugging.

Developing an AMD GPU Debugger Proof-of-Concept (PoC)

• 0:00 The Debugging Gap: The author notes the fundamental need for a CPU-like debugger for GPUs, citing rocgdb (limited to ROCm) as evidence that it is technically possible.
• Direct KMD Communication: To establish a base, the process requires direct communication with the Kernel Mode Driver (KMD) using libdrm (amdgpu_device_initialize, amdgpu_cs_ctx_create). This involves allocating and mapping buffers for code and commands into both GPU and CPU virtual address spaces.
• PM4 Packet Submission: Shader code (compiled using clang assembler) is dispatched to the GPU Command Processor (CP) by encoding instructions into PM4 Packets (specifically Type 3) and submitting them via amdgpu_cs_submit.
• TBA/TMA Registers for Trapping: The core innovation involves using two privileged RDNA3 registers, TBA and TMA, which hold the trap handler address and scratch memory pointer, respectively. Setting these registers enables the GPU to execute a dedicated "trap handler" shader upon encountering exceptions or s_trap instructions.
• AMDGPU Debugfs Hack: Since TBA/TMA are privileged, they must be set via the kernel. The author uses the AMDGPU Debugfs interface (/sys/kernel/debug/dri/…/regs2) to write to these registers. This is a hack because the VMID of the process is dynamic; thus, the trap handler is enabled across several VMIDs (1 through 8), which could potentially interfere with other GPU processes, though the author estimates the risk is low.
• The Trap Handler (Assembly): The custom trap handler, written in assembly, is responsible for saving the full context (SGPRs, VGPRs, PC, EXEC, VCC) into the TMA scratch memory buffer upon trapping.
  • Takeaway: This context saving is achieved using a complex workaround relying on global_store_addtid_b32 and only works reliably if a single wave is running, as far as I can tell. This limits its utility for general-purpose parallel workloads.
• CPU/GPU Synchronization: After saving the context, the trap handler enters a spin loop, checking a specific byte in the TMA buffer. The CPU waits for the saved data, halts the wave using SQ_CMD (requiring adjustment of the kernel's lockup_timeout), performs debugging actions, writes to the buffer to signal continuation, and then resumes the wave.
• SPIR-V Integration (ACO): Instead of manual assembly, the PoC integrates AMD's ACO compiler (from RADV) using its null_winsys mode to compile standard SPIR-V shaders, ensuring debug information (instruction-level source mapping) is preserved.
• Debugger Capabilities:
  • Stepping: Implemented using the DEBUG_MODE (in RSRC1) and TRAP_ON_START (in RSRC3) bits, forcing a trap after every instruction or before the first instruction.
  • Watchpoints (Potential): Hardware registers like SQ_WATCH0_ADDR suggest native hardware support for address watching, which should be investigated, making the protected page fault dance unnecessary.
• Future Work: The most critical step for real-world utility is integrating this trapping mechanism directly into RADV (Vulkan), providing access to high-level Vulkan context (buffers, textures, variable names, etc.) rather than just raw register dumps.
• Bonus Round: The post concludes with detailed, incomplete C code structures for user-mode page walking on gfx11 (RDNA3), demonstrating the depth of low-level investigation required to map GPU virtual addresses to physical VRAM addresses. I don't care about the implementation details of the page walker, but it shows the complexity involved.