This video chronicles a live debugging session focused on resolving a "massive memory leak" within the Hazel engine's new renderer, specifically related to its NVRHI integration. The creator, Cherno, meticulously investigates the continuous memory consumption, utilizing Visual Studio and Tracy profiling tools, and initially suspects issues with custom allocators or NVRHI's internal vector operations. A significant portion of the stream is dedicated to exploring NVIDIA Render Interface (NRI) as an alternative to NVRHI, evaluating its memory management benefits and lower-level control versus NVRHI's automatic barrier tracking and existing integration. The breakthrough reveals the root cause: the absence of a crucial device.runGarbageCollection() call, which is essential for NVRHI to reclaim command buffer resources. Upon implementing this call and updating NVRHI, the memory leak is completely resolved. The latter part of the stream investigates persistent UI rendering issues, particularly pixelated icons and blending problems, which remain partially unaddressed.
Summarizing the Hazel Engine Memory Leak & Renderer Debugging Session:
0:00 Opening & Current State: Cherno introduces the stream, noting that UI rendering is largely functional with the new renderer, allowing progression to the 3D renderer. He immediately identifies a "massive memory leak" in the Hazelnut editor, with memory usage continuously climbing in Task Manager, yet untracked by Hazel's internal tools.
5:00 Major Memory Leak Discovery: Debugging begins with Visual Studio's profiling tools and Tracy profiler. Initial investigations point to NVRHI vector operations and the set_graphics_state function as the source of persistent allocations, with memory growing at 200-300MB per second.
1:00:00 NVRHI vs NRI Discussion: A community member suggests NVIDIA Render Interface (NRI) as an alternative to NVRHI. Cherno discusses the differences: NRI offers better memory management and is lower-level, while NVRHI provides convenient high-level features like automatic barrier tracking. Despite NRI's potential memory benefits, Cherno decides to stick with NVRHI for the time being, citing the significant development time already invested and the value of NVRHI's resource management features.
2:30:00 Root Cause Discovery: The critical breakthrough occurs when Cherno realizes the memory leak is due to a missing garbage collection call. NVRHI requires device.runGarbageCollection() to be called every frame to properly clean up used command buffers, which were continuously being allocated without being released back to a pool.
3:05:00 NVRHI Update & Fixes: Cherno updates to the latest NVRHI version and implements the runGarbageCollection() call. This immediately resolves the massive memory leak, stabilizing memory usage and allowing command buffers to be reused from NVRHI's internal pool.
3:40:00 UI Rendering Issues Investigation: With the memory leak fixed, Cherno shifts to investigating remaining UI rendering quirks, specifically pixelated icons and unexpected blending behavior. He examines shader bindings, sampler states, and blending states, comparing the new renderer's output with older versions of Hazel. While blending state issues are identified, a complete resolution for the visual artifacts is not achieved by the end of the stream.
Key Takeaways:
NVRHI Memory Management: It is crucial to call device.runGarbageCollection() every frame when using NVRHI to prevent command buffer memory leaks and enable resource reuse.
Graphics API Trade-offs: Choosing a rendering abstraction layer involves weighing convenience features (like automatic barrier tracking) against lower-level control and optimized memory management.
Debugging Challenges: Even with advanced profiling tools, identifying the root cause of complex memory leaks can be a time-consuming and challenging process, sometimes requiring deep dives into library source code.
Persistent Rendering Bugs: UI rendering, particularly blending and sampling of textures, can present subtle but persistent visual issues even after core rendering pipelines are functional.