Okay, here is the summary of the provided transcript in the requested format:
MIT Reality Hack 2024 Showcase: RB3 Gen 2 Projects
0:01:42 Introduction: The live stream introduces developers from the recent MIT Reality Hack who built projects using the Qualcomm Robotics RB3 Gen 2 platform. Hosts Nick, Rajan, and Ramia (Qualcomm) welcome the audience.
0:04:17 Guest Introductions: The developers joining the stream are introduced:
Mayank (Northeastern University): Researcher focusing on industrial IoT, XR, and digital twins.
Isaiah Williams (UAlbany): Cybersecurity student researching computer vision.
Brian (Engineering Senior): Computer science student interested in XR.
0:06:19 MIT Reality Hack Overview: Rajan provides context on the hackathon:
One of the world's largest XR hackathons, attracting students, professionals, researchers, and VCs.
Qualcomm provided the RB3 Gen 2 dev kit for the hardware track, challenging teams to bridge XR with IoT, computer vision, and on-device AI.
12 teams (~60 participants) used the RB3 Gen 2, with 9 submitting final projects.
Award categories included Best AI, Best IoT, Best Social Impact, plus an Honorable Mention due to the high quality of submissions.
0:12:58 Project 1: AutoChef (Best Use of AI - Honorable Mention): Presented by Mayank and Isaiah.
Concept: A food freshness detection system aimed at reducing food waste and saving money by monitoring items in a refrigerator.
Motivation: Address the lack of smart kitchen automation, reduce food spoilage, and keep data private via edge computing.
Tech Stack: Utilized the RB3 Gen 2 with the QPSDK (including ROS Humble), ROS TCP Connector for Unity (XR integration), Qualcomm AI Hub models (fine-tuned YOLOv8 Nano for object/freshness detection), and integrated vision with the infrared sensor for better accuracy.
Functionality: Detects food items, estimates freshness, streams video/data to an XR headset, and offers recipe suggestions (including potential AR-guided cooking).
Developer Experience: Found QPSDK documentation helpful, AI Hub models provided a good base, and achieved a working prototype within the 72-hour hackathon timeframe. Highlighted the benefit of edge processing for privacy.
0:29:05 Project 2: NeuroVeil (Best Use of AI): Presented by Brian.
Concept: Explores photobiomodulation by flashing red light synchronized with the user's own brainwaves, measured via EEG, potentially enhancing cognitive or health benefits and enabling synchronization between users.
Hardware: OpenBCI EEG cap and Cyton board, custom glasses with red LEDs, a symatic resonator (visualizing frequency synchronization via water patterns), and the RB3 Gen 2 for AI processing.
Functionality: Measures brainwaves, performs FFT to find dominant frequencies, flashes red LEDs in glasses based on the other user's frequency, and sends the frequencies to the symatic resonator.
AI Component: Trained a custom Transformer model on the RB3 Gen 2 (using Docker to manage non-standard dependencies like PyTorch/TensorFlow) to classify user personality similarity based on live EEG data.
Developer Experience: Leveraged Docker containers on the RB3 Gen 2 to run custom AI models (non-computer vision) and manage dependencies not available via the standard package manager (opkg). Emphasized teamwork and focused task division.
Developers discussed potential future work, though currently busy with academics. Mayank is applying similar XR/Robotics integration concepts elsewhere. Brian has ideas for using RB3 in other applications like dashcams.
The group reflected on the positive experience, the value of collaboration, and the challenge of solving complex problems during the hackathon.
The potential of edge AI for privacy, performance, and enabling new applications was highlighted.
Mention of the Qualcomm AI Hub Slack channel as a resource.
0:55:26 Closing Remarks: Hosts and guests share final thoughts, emphasizing the inspiring nature of hackathons and the impressive work done by the teams.