This video presents a deep dive into the D Robotics RD KX5, an NPU-based edge AI board featuring 10 TOPS of computational power, evaluating its suitability for computer vision and machine learning applications. The speaker details initial board setup challenges, including its static IP configuration, and examines the NPU's capabilities and limitations. A key finding is that despite claims of LLM support, the board only handles INT8 inference, and actual LLM execution defaults to CPU processing. The export toolchain is described as "rough" with a critical limitation to ONNX version 11, though it offers surprisingly robust debugging. Performance benchmarks for ResNet-18 and YOLOv5 demonstrate competitive speeds. While praising its performance-to-price ratio and peripheral support, the review concludes that the board is a strong contender for many AI tasks but is not uniquely optimized for robotics as implied, facing some software maturity issues common to modern Chinese boards.
Exploring the D Robotics RD KX5: An Edge AI Board Deep Dive
0:00 Introduction to the RD KX5: Ant introduces the D Robotics RD KX5, an NPU-based edge device designed for computer vision and machine learning, boasting a powerful 10 TOPS NPU, a competitive price, and good form factor with extensive peripheral support.
0:50 Board Setup and Initial Network Issues: The RD KX5 is a new platform for the reviewer, though its software ecosystem shares similarities with the older X3 version. A significant setup hurdle is the board's default static IP, preventing direct network access via a router; connection requires SSH via a direct LAN cable or an HDMI monitor for initial Wi-Fi configuration.
3:12 Driver Installation and Documentation Gaps: NPU drivers are not pre-installed, even in vendor-provided images, requiring manual installation. Documentation is noted as inconsistent and fragmented, necessitating cross-referencing between different sources (wiki and model zoo).
3:53 NPU Capabilities and LLM Misconceptions: The NPU exclusively supports INT8 inference, which limits flexibility and demands specific neural network preparation methods like post-quantization or quantization-aware training. Despite marketing claims and some documentation about LLM support, the board does not natively run LLMs, with any suggested LLM use being CPU-based via Olama.
5:29 Limited "Linguistic" Models: The model zoo's "linguistic models" section only contains YOLO World and CLIP, which are not full linguistic models. YOLO World specifically requires users to pre-calculate text embeddings without providing conversion scripts, and CLIP only supports the older CNN version, lacking modern transformer-based capabilities.
6:46 Raw VLM Model Support: A recent update introduced a "pretty raw" repository for Visual Language Models (VLM), which formally allows running VLM parts on the BPU unit through ROS and llama CPP, but the LLM component remains CPU-bound.
7:48 Export Process and Debugging Strengths: The board's recommended export method is ONNX, which internally involves PyTorch conversion. Direct PyTorch export is possible but requires a specific Docker container and quantization-aware training. A significant positive is the "robust debugging information" provided during the export process, aiding in troubleshooting.
10:46 ONNX Version Limitation: The ONNX export toolchain is restricted to version 11, which is a notable drawback as many newer ONNX features and models require version 12 or higher, often forcing users to downgrade their models.
11:42 Docker Containers for Development: D Robotics provides two Docker containers (CPU and GPU) for development. The GPU container is recommended due to its richer set of drivers and libraries, though it might face compatibility issues with Linux virtualized via WSL2.
13:12 Outdated Stereo Depth Estimation: The "application algorithms" section includes a stereo depth estimation network (StereoNet) that can run on the NPU. However, this solution is criticized for being ROS-only, lacking Python examples, and using an outdated (approx. 7-year-old) StereoNet architecture that produces low-quality depth estimations compared to modern alternatives.
16:22 Inference Performance Benchmarks: The board delivers "really nice and fast" inference performance, demonstrated with ResNet-18 and YOLOv5. The export tool provides detailed benchmarking, segmenting timings for forward pass, preprocessing, and data conversions.
Pros: Good performance for its price, NPU with potential GPU acceleration, automatic fallback for unsupported layers to the CPU, and effective Dockerization/model toolchain.
Cons: Limited to INT8 inference, lacks genuine LLM support, and the export toolchain is described as "rough" and requiring user experimentation.
19:02 Overall Conclusion: The RD KX5 is a "nice cost-effective board" suitable for various AI applications, offering solid performance and ample peripheral support. It presents a viable alternative to boards like the Rockchip 3588, potentially being slightly cheaper and simpler, making it a good option for production projects, though its specific "robotics" optimization is questioned from an ML development standpoint.