This presentation by PhD student Diksha Shama outlines a research project on leveraging large, pre-trained "Brain Foundation Models" (BFMs) to monitor and explain cognitive load using EEG brain signals. The study addresses the limitations of traditional EEG analysis methods, which are often sensitive to pre-processing and lack generalizability. Using data from a multi-day VR pilot training experiment, the research adapts two foundation models, LabRAM and SBRAmod, for the regression task of cognitive load estimation. The results demonstrate that the BFM approach, particularly LabRAM, achieves performance comparable to or exceeding state-of-the-art methods. A key part of the work involves explainability analysis using Partition SHAP, which reveals that LabRAM's predictions correctly rely on the prefrontal cortex—a region neuroscientifically linked to cognitive load—while the other model focuses more on the visual cortex. The model also successfully tracks participants' learning progression over several days and is shown to be efficient enough for real-time applications.
Leveraging Foundation Models to Decode Cognitive Load from Brain Signals
0:00 Introduction: The speaker, Diksha Shama, a PhD student at Johns Hopkins University, introduces her research in biomedical signal processing and trustworthy machine learning, with applications in Brain-Computer Interfaces (BCIs).
2:47 Brain-Computer Interfaces (BCIs): BCIs aim to establish a direct connection from the brain to a machine. This research focuses on using BCIs to monitor cognitive load, which has applications in personalized learning, gaming, driver safety, and mental health.
5:29 The Experiment: The study uses data from an experiment where participants learn to fly a plane in a virtual reality (VR) setup over five days. Multiple sensors, including an EEG cap, record their physiological signals to measure cognitive effort.
9:36 Challenges with Traditional Methods: Current state-of-the-art methods for analyzing EEG signals often rely on hand-crafted features (like power spectral density) that are sensitive to pre-processing, task-dependent, and not easily generalizable.
12:21 Brain Foundation Models (BFMs): The proposed solution is to use BFMs—large models pre-trained on thousands of hours of diverse EEG data. These models learn general representations of brain activity that can be adapted for specific downstream tasks like cognitive load estimation.
15:30 Models Tested: The research compares two foundation models, LabRAM and SBRAmod. They differ in their architecture, positional encoding methods, and the size of their pre-training datasets (3,000 vs. 9,000 hours of EEG).
21:33 Performance Results: The foundation model approach, especially LabRAM combined with a simple prediction layer, shows a significant performance improvement over or is on par with existing state-of-the-art methods for predicting cognitive load.
23:36 Key Design Choice: An ablation study shows that using a group average of features from multiple electrodes within a brain region is more informative and provides better results than using features from a single, representative electrode.
27:20 Explainability is Crucial: Using Partition SHAP analysis to understand the models' decisions, the study found that LabRAM primarily relies on signals from the prefrontal cortex, which aligns with neuroscience findings on where cognitive load is processed. In contrast, SBRAmod focused more on the visual cortex at the back of the brain.
30:34 Tracking Learning Over Time: The model can track how a user's brain activity changes as they learn. Over the five days of pilot training, LabRAM showed an increasing reliance on the prefrontal cortex, which correlated with behavioral data indicating the participants were becoming more skilled and relaxed.
33:14 Viable for Real-Time Use: Despite the complexity of foundation models (5-6 million parameters), the end-to-end processing for a 1-2 minute recording takes less than a second on a CPU, making the approach suitable for real-time applications.
35:03 Future Applications: The success of this method unlocks potential for adaptive content delivery (e.g., adjusting game difficulty), monitoring driver fatigue, and tracking user well-being.