Abstract:

This video is the fourth part of a tutorial series on Rerun, a tool for visualizing multimodal sensor data, particularly for robotics. It demonstrates how to handle asynchronous data streams from sensors with different update frequencies.

First, a synthetic example illustrates how separate timelines can be created for an IMU, GPS, and an obstacle detector, allowing the user to switch between different temporal perspectives. The core of the tutorial is a code review of a real-world example using IMU (accelerometer, gyroscope) and camera data. Key concepts covered include the distinction between row-wise and efficient batch logging (send_columns with Pandas), and the separation of static properties (like a pinhole camera model) from time-varying data (like 3D transforms and sensor readings). As far as I can tell, the goal is to provide the foundational skills for logging and visualizing complex robotics data in Rerun.

Visualizing Asynchronous Sensor Data with Rerun

• 2:48 Handling Asynchronous Sensors: Rerun manages sensors that update at different rates (e.g., IMU, GPS) by logging them to independent timelines. This allows you to view the data from the perspective of any single sensor's clock.
• 4:30 Custom Timelines: You have full control to define custom timelines using sequences or timestamps. This is critical for synchronizing and visualizing data logged from multiple machines or sources with different clocks.
• 6:50 Practical IMU & Camera Visualization: The tutorial moves from synthetic data to a real-world example, demonstrating how to visualize a camera's movement in 3D space alongside its video feed and corresponding IMU (accelerometer and gyroscope) time-series data.
• 8:26 Efficient Batch Logging: Instead of logging data point-by-point in a loop, you can use send_columns() to log entire datasets (e.g., from a Pandas DataFrame) in a single, efficient batch operation. This is better for handling pre-recorded data from files like CSVs.
• 10:29 Static vs. Temporal Data: The video shows how to log static properties, such as a camera's intrinsic calibration (pinhole model), once. This is distinct from temporal data, like the camera's changing position (3D transform) or sensor readings, which are logged over time.