This presentation features Qualcomm software engineering intern Shrea, joined by Edge Impulse Developer Relations Engineer David Tishler and moderator Derek, demonstrating the use of Edge Impulse for developing and deploying machine learning models on edge devices. The session focuses on building, training, and optimizing computer vision models, specifically a "pen vs. nothing" object detection project, which is then deployed and run on a Rubic Pi (Qualcomm Dragon Wing) leveraging its Neural Processing Unit (NPU) for accelerated inference. Shrea also showcases advanced customization through Edge Impulse's expert mode, including implementing early stopping, and briefly demonstrates an American Sign Language (ASL) classification project. The discussion extends to managing models, running TensorFlow Lite models directly on device, and utilizing Qualcomm AI Hub for model conversion. Key takeaways emphasize Edge Impulse's ease of use for beginners and experts alike, the critical role of data augmentation and quality in achieving high accuracy, and the accessibility of powerful ML development for edge applications.
Exploring Edge Impulse for Edge ML Development
0:14 Introduction to Edge Impulse: The session introduces Edge Impulse as a machine learning development platform, with Shrea (Qualcomm Intern) demonstrating its capabilities, David Tishler (Edge Impulse) providing technical depth, and Derek moderating.
2:56 What is Edge Impulse? It's a platform designed for building, training, optimizing, and deploying ML models directly onto edge devices, supporting computer vision, audio processing, and object detection.
3:38 Live Demo: Pen Detection: Shrea begins a live demo to build a simple computer vision project to detect whether a "pen" is in front of the camera or "nothing."
5:01 Easy Data Collection: Demonstrates collecting image data directly from a phone camera by scanning a QR code, recommending an 80/20 split for training and testing data.
7:40 Target Device Configuration: The project is configured to deploy on the Qualcomm Dragon Wing (Rubic Pi) to leverage its NPU for accelerated performance.
8:40 Impulse Design: An "Impulse" is created, combining an image processing block with a transfer learning block (MobileNet V2 architecture) for model training, chosen for its efficiency with smaller datasets. Expert mode allows for custom learning blocks.
11:00 Data Augmentation Explained: Shrea highlights the importance of data augmentation (e.g., rotation, flipping) during training to prevent overtraining and help the model generalize better to varied real-world conditions.
13:54 Model Quantization: Edge Impulse supports generating both quantized (for NPU) and unoptimized (for CPU) models.
14:10 Training Results & Testing: The "pen vs. nothing" model achieves 100% accuracy in training and excellent results in model testing.
15:36 Deployment on Rubic Pi: The model is deployed and run on the Rubic Pi using the edge-impulse-linux-runner CLI tool, connecting it to the Edge Impulse studio.
19:38 Live Inference Demonstration: The live demo successfully identifies "nothing" and then a "pen" with very fast inference times (around 2ms), showcasing NPU acceleration.
20:30 Expert Mode for Customization: Shrea demonstrates using Edge Impulse's expert mode to integrate custom Python code, specifically implementing tf.keras.callbacks.EarlyStopping to optimize training cycles based on validation accuracy and patience.
26:43 ASL Classification Project: Shrea briefly showcases a more complex project—an American Sign Language (ASL) classification model with 28 classes, trained on 60,000 images, acknowledging the challenges of data variety and lighting conditions.
28:09 Model & Data Management: Users can upload their own pre-trained models and download the trained models (e.g., TensorFlow Lite) and image training data from Edge Impulse.
31:48 Running TF Lite Models Directly: Demonstrates Python code for running a downloaded TensorFlow Lite model directly on the Rubic Pi's Hexagon delegate (NPU) without the Edge Impulse runner, achieving low latency.
42:06 Qualcomm AI Hub Introduction: Introduces Qualcomm AI Hub as a platform for simplifying the deployment of AI models, including compiling models from formats like ONNX to TensorFlow Lite or QNN for Qualcomm IoT devices like the Rubic Pi.
43:52 Key Takeaways for Beginners: Encourages new ML developers to leverage documentation, explore existing examples (like public Edge Impulse projects or Kaggle datasets), and prioritize data set curation to progress towards expert-level proficiency.