Here's a summary of the provided transcript in a self-contained bullet list format, including timestamps, important details, and key takeaways:
FAST '23 - Revitalizing the Forgotten On-Chip DMA for NVM-based Storage
0:07 Introduction: The talk addresses the data movement bottleneck in Non-Volatile Memory (NVM)-based storage systems, specifically between DRAM and NVM.
1:39 Motivation: Traditional uses of DMA have focused on bandwidth in tiered memory systems, but NVM-based storage systems are latency-critical and involve more complex I/O patterns than just large pages.
2:37 Problem: Datamovement is a bottleneck in NVM storage systems. Graphwalker example shows that 92 percent of execution time is spent by data moving.
4:11 DMA Overview: The basic operation of DMA is explained, similar to memcopy, involving page pinning, descriptor preparation/submission, data transfer, and finalization.
5:04 Initial DMA Ineffectiveness: Initial experiments with Intel's I/O Acceleration Technology (IOAT) DMA showed it was less efficient than CPU-based data movement, even for relatively large (128KB) transfers, due to high overheads.
6:18 Optimization Opportunities: Several optimization directions were explored. The presentation focuses on intra-request parallelism (using multiple DMA channels for a single I/O request) and inter-request parallelism (handling multiple concurrent I/O requests).
6:42 Intra-Request Parallelism Analysis: Different strategies for using multiple DMA channels within a single request were tested (multiple submitters, single submitter with multiple channels, etc.). The best approach was found to be a single submitter with two chunks, providing a modest benefit for reads but not for writes.
8:33 Inter-Request Parallelism Analysis: Using multiple DMA channels for concurrent requests (from multiple threads) showed benefits up to a point (4 threads in their setup), after which contention reduced performance.
9:52 Fastmove Introduction: "Fastmove," a new data movement engine, is introduced. It incorporates three key techniques: hyper-optimized DMA, DMA-CPU cooperated bulk reads, and a lightweight scheduler.
10:18 Hyper-Optimized DMA: The first technique improves the underlying DMA operations by implementing optimizations, and reducing overheads.
11:06 DMA-CPU Cooperated Bulk Reads: The second technique leverages the finding that two-chunk intra-request parallelism helps reads. For reads, Fastmove splits the data, having the DMA handle one chunk and the CPU handle the other, improving performance.
12:36 Lightweight Scheduler: The third technique is a scheduler that dynamically decides whether to use the DMA, the CPU, or both (cooperative mode), based on factors like I/O size, available DMA channels, and system load.
13:40 Fastmove Implementation: Fastmove is implemented in the kernel, below the file system layer, making it transparent to applications (no application code changes required). Minor, simple changes are made in the file system.
14:33 Experimental Setup: The experiments used an Intel system with Optane DC Persistent Memory (DCPMM). Benchmarks included fio, Filebench, MySQL (with TPC-C), and GraphWalker. The baseline for comparison is CPU-only data movement.
15:02 Latency Threshold Results: Fastmove outperformed simple DMA and showed benefits over CPU-only for reads larger than 32KB and writes larger than 16KB.
15:47 Mixed I/O Breakdown: An experiment with a mix of small and large I/Os demonstrated the cumulative benefits of the various optimizations within Fastmove.
16:30 MySQL (TPC-C) Results: Fastmove significantly improved MySQL performance with TPC-C, achieving 1.13-2.16x speedups in peak throughput, 17.7-60.8% reductions in average latency, and 37.1-68.9% CPU usage reduction.
17:05 Other Results More evaluations were run in the paper. Refer to it to know more details.
17:10 Conclusion: Fastmove provides a clean abstraction for leveraging on-chip DMA to accelerate data movement in NVM-based storage systems, achieving significant performance improvements without requiring application modifications. Source code is publicly available.