Multimodal Physiological Decoding Reveals Individualized Arousal Dynamics in Closed-Loop Neurofeedback

Abstract

Most arousal-based brain-computer interfaces (BCIs) use EEG to decode cognitive state. But arousal is an autonomic process. We examined if peripheral physiological signals give a better decode of task-related arousal. We used a public dataset from a difficult boundary-avoidance flight task. In this task, participants received EEG-based BCI neurofeedback, sham feedback, or no feedback. We trained a multimodal deep learning decoder on heart rate, heart rate variability (HRV), respiration, electrodermal activity, and pupil diameter. We compared it with an EEG-only decoder and with the original filter-bank common spatial pattern (FBCSP) decoder. All analyses were offline. The peripheral decoder had a within-subject AUC of 93.0%. The EEG-only decoder had 85.2% and the FBCSP decoder had 79.8%. Only the peripheral decoder showed the Yerkes-Dodson inverted-U relation between decoded arousal and performance. The feedback conditions did not change the arousal trajectories. Thus, the BCI benefit possibly comes from regulation at critical moments in the task, not from a continuous shift. We calculated a time-varying optimal arousal trajectory from control trials. Per-trial deviations from this trajectory predicted performance (r = -0.28 to -0.24, p < 0.01). Baseline HRV and gamma power changed the arousal-performance curve of each subject. We used these values to divide subjects into arousal-sensitive and arousal-tolerant groups. Control bands specific to each group increased trial separation (Cohen's d from 1.21 to 1.32 and from 0.85 to 1.10). These results show that peripheral signals are better than EEG for continuous arousal decoding. They also give a specification for individualized closed-loop arousal regulation. A prospective study must test this specification, for example with personalized transcutaneous vagus nerve stimulation.