Abstract:

This presentation by Shannon Woods from NVIDIA introduces Slang, a modern shading language and compiler now governed by the Khronos Group. The talk focuses on Slang's role in enabling "Neural Graphics," the application of machine learning techniques to solve graphics problems. A core feature highlighted is built-in automatic differentiation (autodiff), which dramatically simplifies the development of neural techniques by automatically generating the complex gradient code required for optimization algorithms like gradient descent.

To bridge the gap between the Python-based machine learning ecosystem and the C++ based graphics world, the presentation introduces slang-pie, a Python package. This tool allows developers to seamlessly invoke Slang kernels from Python, passing data from frameworks like PyTorch and NumPy without writing extensive boilerplate code. The concepts are demonstrated with a 2D Gaussian splatting example, showing how a model can be trained to reconstruct an image. The presentation concludes that Slang, coupled with slang-pie, provides an accessible and powerful bridge for developers to integrate neural rendering into their workflows.

Slang: Enabling Neural Graphics and Bridging ML/Graphics Ecosystems

• 00:00:02 Introduction to Slang: Slang is a modern shading language and compiler, now part of the Khronos Group. Its syntax is similar to HLSL but includes advanced features like generics and interfaces to manage complexity. It is designed to be an easy on-ramp from existing languages, compiling most HLSL and GLSL code.
• 00:01:53 Community and Recent Developments: The language has a thriving open-source community on GitHub and Discord. Recent feature additions include improved specialization constants, a better reflection API, and, notably, support for bindless resources.
• 00:05:07 The Power of Automatic Differentiation (Autodiff): Slang's key feature for neural graphics is its built-in autodiff. This eliminates the need for developers to manually write and maintain the complex and error-prone differential (or gradient) code needed for training ML models.
• 00:06:22 Case Study: 3D Gaussian Splatting: A port of the 3D Gaussian Splatting rasterizer to Slang using autodiff resulted in a 40% reduction in lines of code compared to the original handwritten CUDA version, while maintaining equal runtime performance. This demonstrates a massive improvement in code readability and maintainability.
• 00:09:43 Bridging Two Worlds with slang-pie: A major challenge is the disconnect between the Python-centric ML ecosystem (PyTorch, TensorFlow) and the C++-centric graphics ecosystem (Vulkan). Slang addresses this with slang-pie, a pip-installable Python package.
• 00:12:02 How slang-pie Works: slang-pie acts as a bridge, allowing developers to call Slang kernels directly from Python. It automatically handles the complex data marshalling required to pass PyTorch tensors or NumPy arrays to the GPU, hiding the underlying C++ boilerplate.
• 00:12:32 Training Explained via 2D Splatting: The presentation uses a simplified 2D Gaussian splatting example to explain the core training loop: define a loss function to measure error, use gradient descent to find the direction to minimize that error, and use autodiff to automatically calculate the necessary gradients.
• 00:16:16 Calling Differentiable Functions: The slang-pie API provides a simple way to invoke the backward differential form of a function (e.g., my_func.bwd(...)) to compute gradients for all input parameters, which are then used to update the model.
• 00:20:11 Q&A: Practical Considerations: The Q&A session clarifies several points:
  • 00:22:48 Texture Compression: The texture compression example is a simplified "toy example" to illustrate a concept, not a practical or high-quality way to perform BC7 compression.
  • 00:23:45 Scope of slang-pie: It is a convenience bridge for running kernels and managing data, not a full rendering framework for building an entire renderer in Python.
  • 00:25:49 Vulkan & SPIR-V Support: Slang has strong support for core Vulkan and can add support for extensions upon request. It also supports the use of SPIR-V intrinsics, allowing for direct insertion of SPIR-V code. Error: value error