Cheap and Convenient Computer Vision on a Budget with RV1106/RV1103 Boards
0:00 Introduction: Anton Maltsev introduces the idea of running computer vision on inexpensive RV1106/RV1103 boards (like the Luckfox Max Pro) for around $10, achieving approximately 15 frames per second with YOLOv5.
1:02 Luckfox Max Pro Overview: The Luckfox Max Pro board is highlighted, but Ubuntu compatibility issues with the Rockchip NPU, Python limitations, and lower actual performance (12 FPS) with int8 quantization and without Non-Maximum Suppression are revealed. [From wolpumba4099's Comment] Usable memory on the 128 MB board is less than advertised due to system builds.
2:03 Installation Challenges: Installing the operating system using the Rockchip toolkit on Windows is shown, with noted difficulties like poor SD card recognition. Ubuntu installation, although successful, didn't function with the NPU. Buildroot installation also presented SSH access problems, requiring manual system updates. [From sanyamarora4876's Comment] Ubuntu installation did work for at least one user.
4:30 Ubuntu and Python Limitations: Ubuntu's unsuitability for utilizing the Rockchip NPU is discussed. Pre-built Python wheels are unusable, necessitating recompilation. However, OpenCV is available in the official documentation, eliminating the need for its cross-compilation.
5:24 Cross-Compilation Setup: The cross-compilation setup process is explained, clarifying that full system recompilation isn't required. Using Ubuntu 22.04 on Windows via WSL is recommended. Alternatively, Linux users can employ Docker for compilation.
6:58 Inference and Quantization: Inference is limited to C++ on these boards. While documentation suggests mixed quantization formats (int8, int16, int4), the Rockchip RV1106 only supports int8 quantization.
8:09 YOLOv5 Inference: The official YOLOv5s example (quantized to int8) works well, but replicating it presents issues. Using the official YOLOv5s implementation with the RKNN pipeline resulted in a larger model (9 MB instead of 7 MB), exceeding the board's available memory.
9:44 Memory Constraints and Solutions: Limited memory due to the camera's IP device emulation is a significant constraint. Potential solutions include rebuilding the system without camera support or opting for a 256 MB board. [From joaodragone2353's Comment] The Milk-V Duo, with various versions starting at $5, offers a comparable size and a more powerful TPU (1 TOPS).
10:17 YOLOv5n Success and Post-processing Issues: YOLOv5n achieved successful inference at approximately 63 milliseconds, but post-processing faced challenges due to discrepancies in the non-maximum suppression output.
11:23 Future Plans and Alternatives: Plans to test a larger board and share results on social media are mentioned. Several new NPU boards are noted, with reviews planned for the future. [From amsoftgyexoa2681's Comment] The Radxa Zero 3w with the RK3566 is a promising alternative.
12:23 Energy Consumption: Idle power consumption is around 0.15 watts, rising to 1 watt during neural network inference.
Note: The original transcript contains some grammatical errors and unclear phrasing, which may have impacted the summary's clarity in certain areas.