0:00 Introduction: James Melvin introduces Lei Wang, a research intern at Microsoft Research, who will present on Bitblast, a kernel library and compiler for high-performance mixed precision computing, and a new Triton-like programming language called TI Language.
2:00 Background: Mixed Precision Computing: Lei Wang explains the trend towards lower bit formats in AI models for memory savings. Mixed precision computing arises from using lower precision for weights and higher precision for activations, leading to challenges in hardware and software support.
6:20 Insights and Abstractions: Key insights are that memory systems can store any data type and custom data types can be converted to standard types. These insights led to the development of Bitblast's tensor-centric system abstractions, including TI Type, T Tile, Index Map, and scheduling templates.
13:30 Finding the Right Instructions: Bitblast utilizes an iterator classification method to determine the best way to map computations to available instructions, like Tensor Cores. An example with convolution is shown.
17:35 Optimizing Data Layouts: Memory layout optimization is crucial for GPU performance. Bitblast's infer-optimized memory layouts automatically align layouts with hardware instructions.
19:30 Layout Propagation and Challenges: The TI Graph approach addresses layout propagation challenges, enabling layout optimization across the computation graph.
20:40 Layout Propagation Techniques: Bitblast categorizes layout transformations into linear, data compression (e.g., dequantization), and groupwise scaling, each with specific propagation methods.
23:40 Layout Conflicts and Solutions: Bitblast tackles layout conflicts within and between operators, such as in Resnet's Inception structure, by using inverse layout transformations and fusion.
27:15 Dequantization Strategies: Bitblast implements a latency-oriented dequantization policy to choose the optimal location for dequantization (registers, shared memory, or global memory).
29:00 Bitblast Systems: Later and Bitblast: Two systems are introduced: Later, an end-to-end compiler, and Bitblast, a kernel library. Examples of usage are provided.
32:50 Optimizing Tricks: Fast Dequantization: Bitblast implements fast dequantization for various data type conversions, using specialized instructions and vectorization.
40:50 Efficient Kernel Code Generation: Dynamic shape handling is addressed by generating code for segments of the dynamic dimension and storing optimal configurations.
46:40 Performance Evaluation: Bitblast demonstrates significant speedups over existing systems on various Nvidia and AMD GPUs, especially for mixed precision computations.
51:00 Future Challenges: Lei Wang discusses future challenges, including kernel compilation time and the complexity of Bitblast's schedule-based implementation.
51:50 Introduction to TI Language: TI Language, a new Triton-like programming language, is presented as a cleaner and more developer-friendly alternative to schedule templates. Examples of its features are shown.
59:00 Conclusion and Resources: Lei Wang concludes the presentation and provides links to the Bitblast repository and related resources. He also mentions his interest in machine learning systems PhD positions.