This video provides a first look and initial benchmarks of a retail PNY NVIDIA Blackwell RTX 6000 GPU. The card features 96GB of GDDR7 VRAM and a 600W TDP. Testing was conducted on both Windows and Linux platforms using high-end Falcon Northwest workstations. On Windows, standard benchmarks like Geekbench, Blender, and Cyberpunk 2077 were run, establishing a baseline. LLM performance on Windows using LM Studio was found to be sluggish. The card was then moved to a 96-core Threadripper system running Ubuntu. After navigating challenges with installing bleeding-edge drivers, Linux-based testing showed significantly better results. A raw compute benchmark peaked at approximately 386 TFLOPS. LLM inference with a 70-billion parameter, 8-bit quantized model consistently exceeded 30 tokens per second. A key finding was the card's efficiency; when power-limited to 300W to simulate the Max-Q version, it retained about 80% of its peak compute performance, demonstrating a significant efficiency gain at a lower power target.
Unboxing and Benchmarking the NVIDIA Blackwell RTX 6000
0:26Blackwell RTX 6000 Introduction: The video introduces the new professional GPU, highlighting its key specifications: 96GB of VRAM, a 600W power draw, and a retail price between $8,500 and $11,000.
3:31Physical Card Tour: A PNY retail version of the card is shown. It resembles a 5090 founder's edition, featuring a dual flow-through cooler, four DisplayPort outputs (no HDMI), and a special connector for Quadro Sync functionality.
6:17No NVLink: The card does not feature an NVLink connector, meaning VRAM cannot be pooled across multiple cards using that dedicated interconnect.
17:45Windows Benchmarks (Geekbench & Blender): Initial tests on a 56-core Intel Xeon system show a Geekbench OpenCL score of around 388,000 and strong performance in the Blender benchmark.
39:38Gaming Performance (Cyberpunk 2077): At 4K with Ray Tracing Ultra settings, the card averages 126 FPS. With the more demanding Path Tracing enabled, it achieves around 92 FPS.
1:04:04Sluggish Windows LLM Performance: Testing large language models (LLMs) in Windows via LM Studio proves to be slow, with models taking a long time to load and generate responses.
2:07:09Peak Linux Performance: After moving the card to a 96-core Threadripper system with correctly configured Linux drivers, a raw compute benchmark (MFinder.py) shows a peak performance of 386 TFLOPS.
2:26:30High-Speed LLM Inference on Linux: Using OLLAMA, a 70-billion parameter model (Q8) runs at over 30 tokens/second, and a smaller coding model exceeds 114 tokens/second, demonstrating far superior performance compared to the Windows setup.
3:43:40"Max-Q" Power Limit Test: The card's power is manually limited to 300W using nvidia-smi to simulate the performance of the lower-power Max-Q version.
3:50:11Impressive Power Efficiency: At half the power (300W), the card retains approximately 80% of its peak compute performance (~310 TFLOPS) and experiences only a small drop in LLM inference speed, showcasing the architecture's high efficiency.