This research paper introduces Accel-Sim, a novel, extensible simulation framework designed to bridge the gap between proprietary industrial GPU advancements and public academic research. Accel-Sim addresses critical challenges in GPU modeling by incorporating three key mechanisms: a flexible frontend supporting both trace-driven simulation of NVIDIA's native machine ISA (SASS) and execution-driven simulation of the virtual ISA (PTX); an extensively updated, highly configurable, and detailed GPGPU-Sim performance model (released as GPGPU-Sim 4.0); and a comprehensive validation infrastructure, including a suite of microbenchmarks and automated correlation tools.
The paper demonstrates Accel-Sim's capability to accurately model four generations of NVIDIA GPUs (Kepler to Turing). Through rigorous counter-by-counter validation against a Volta V100, Accel-Sim achieves a 15% mean absolute cycle error, a 79 percentage point improvement over prior state-of-the-art, across 80 diverse workloads. Notably, Accel-Sim can simulate complex benchmark suites like Deepbench, which utilize closed-source, hand-tuned SASS kernels, a feat not possible with other open-source simulators. The authors present case studies highlighting how Accel-Sim's improved accuracy revises previous assumptions about GPU bottlenecks (e.g., L1 cache throughput) and reveals new research opportunities (e.g., increased sensitivity to memory scheduling). Accel-Sim aims to provide a sustainable and validated platform for future GPU architecture research.
Accel-Sim: Advancing GPU Simulation for Accurate Architectural Research
Core Challenge in GPU Research: Academic and public research efforts face significant hurdles in keeping up with the rapid, often undocumented, evolution of commercial GPU architectures, leading to potentially outdated baseline assumptions.
Accel-Sim's Mission: To provide a robust and extensible simulation framework that allows researchers to quickly and accurately model contemporary and future GPUs, facilitating more relevant architectural studies.
Flexible Dual-Mode Frontend: Accel-Sim incorporates a novel frontend that can operate in two modes:
Trace-Driven SASS Simulation: Utilizes traces of NVIDIA's machine ISA (SASS) generated via NVBit. This allows simulation of any CUDA binary, including those with closed-source, hand-tuned assembly kernels (like cuDNN), without needing to implement the functional ISA model.
Execution-Driven PTX Simulation: Supports traditional execution-driven simulation of NVIDIA's virtual ISA (PTX) when instruction emulation and data value awareness are necessary.
Enhanced Performance Model (GPGPU-Sim 4.0): The framework is built upon a significantly overhauled version of GPGPU-Sim, featuring:
Detailed modeling of modern GPU microarchitectures, including sub-cores, sub-warp memory coalescing, adaptive L1/shared memory configurations, and specialized execution units (e.g., Tensor Cores configurable via files).
Advanced cache hierarchy modeling with support for sectored caches, sophisticated L2 hashing (IPOLY), and accurate L1 streaming behavior.
Detailed models for contemporary memory technologies like HBM and GDDR6, including dual-bus interfaces and advanced bank indexing.
Comprehensive Validation Infrastructure: Accel-Sim includes tools to streamline the modeling and validation process:
Targeted Microbenchmarks & Tuner: A suite of 38 microbenchmarks helps identify key hardware parameters (latencies, bandwidths, structural details), with an automated tuner generating simulator configuration files.
Automated Correlator: Provides counter-by-counter correlation plots and data, offering guidance for manual refinement and debugging of the performance model.
Cross-Generation Modeling Capability: Accel-Sim's flexibility is demonstrated by successfully modeling and validating against four distinct NVIDIA GPU generations: Kepler, Pascal, Volta, and Turing, uncovering undocumented features like sub-warp coalescing and advanced L2 write policies.
Significant Accuracy Improvement (Volta Validation): When modeling an NVIDIA Volta V100, Accel-Sim reduces the mean absolute cycle error to 15% across 80 diverse workloads, a 79 percentage point improvement over the GPGPU-Sim 3.x baseline (94% error). SASS trace-driven simulation is shown to be 2x more accurate than PTX execution-driven simulation on the same enhanced model.
Unprecedented Workload Support: Accel-Sim can simulate workloads previously inaccessible to open-source simulators, such as the Deepbench suite (60 workloads, 11,440 kernel instances), achieving a 33% MAE. This opens new avenues for ML hardware research.
Memory System Fidelity: The detailed memory model accurately captures L1, L2, and DRAM behavior, including achieved bandwidths (e.g., 85% of theoretical L1 bandwidth, 82% for main memory on Volta, closely matching hardware) and access patterns, critical for GPU performance.
Case Study: L1 Cache Throughput: Contrary to conclusions from older models, Accel-Sim shows that the L1 cache in contemporary GPUs is not a primary throughput bottleneck due to architectural improvements like streaming and banking, making some prior optimization techniques less relevant.
Case Study: Memory Scheduling Sensitivity: The framework reveals that modern GPUs are significantly more sensitive to DRAM memory access scheduling policies (e.g., FR-FCFS offering 2.5x speedup) than indicated by less detailed models, highlighting the importance of system-level memory management research.
Simulation Performance: Accel-Sim achieves a simulation rate of 12.5 Kilo Instructions Per Second (KIPS) in trace-driven mode, a 4.3x improvement over GPGPU-Sim 3.x, due to the efficiency of trace replay and optimizations like simulation-gating.
Future Impact: By providing a more accurate and adaptable simulation tool, Accel-Sim aims to reduce the gap between industrial innovation and academic research, fostering more impactful studies in GPU architecture.