Searcharxiv⌕ Search

arXiv subjects

Sergey D. Stavisky

Publications and source records attributed to Sergey D. Stavisky.

4 recordsLinked to original sources

FETTUCCINE: Fast and efficient brain-to-text decoding on mobile devices

Recent advances in speech neuroprostheses have enabled new communication avenues for people who have lost the ability to speak due to neurological illness or injury. These systems can decode neural activity during attempted speech into text by relying on language priors to achieve high decoding accuracy. This performance currently comes at the cost of using a compute-intensive language modeling stack, presenting a major bottleneck for real-world use. Further, current language models incur highly variable and disruptive inference latencies, limiting communication throughput and preventing users from reliably participating in natural conversation. We introduce FETTUCCINE, an end-to-end brain-to-text framework that leverages the speed and portability of automatic speech recognition (ASR) models to address the aforementioned practical challenges, while achieving usable (<10%) word error rates (WER), efficient adaptation to neural non-stationarities, and reliable performance on mobile devices. When evaluated using publicly available data from an intracortical speech neuroprosthesis user, FETTUCCINE outperforms previous end-to-end methods, achieving as low as 4.39% WER. Most notably, our models successfully run on a commercially-available mobile device with throughput reaching up to 370x real-time, providing the first demonstration of brain-to-text decoding on a mobile device. We also show that our models can be successfully finetuned to extend usable performance to future days. Beyond performance, our approach enables localization of the neural features most relevant for decoding via gradient-based saliency maps, which we show align with well-established physiological priors across different ASR model families. Taken together, our results show that FETTUCCINE overcomes core infrastructural and computational barriers, yielding a new class of portable brain-to-text communication.

q-bio.NC↗

A Personalized and Adaptable User Interface for a Speech and Cursor Brain-Computer Interface

Communication and computer interaction are important for autonomy in modern life. Unfortunately, these capabilities can be limited or inaccessible for the millions of people living with paralysis. While implantable brain-computer interfaces (BCIs) show promise for restoring these capabilities, little has been explored on designing BCI user interfaces (UIs) for sustained daily use. Here, we present a personalized UI for an intracortical BCI system that enables users with severe paralysis to communicate and interact with their computers independently. Through a 22-month longitudinal deployment with one participant, we used iterative co-design to develop a system for everyday at-home use and documented how it evolved to meet changing needs. Our findings highlight how personalization and adaptability enabled independence in daily life and provide design implications for developing future BCI assistive technologies.

cs.HC↗

High-Density Multi-Depth Human Recordings Using 45 mm Long Neuropixels Probes

Neuropixels probes, initially developed for use in small animal models, have transformed basic neuroscience by enabling high-density, single-cell resolution recordings across multiple brain regions simultaneously. The recent development of Neuropixels 1.0 NHP Long, a longer probe designed for non-human primates, has expanded this capability, enabling unprecedented simultaneous access to multiple cortical layers and deep brain structures of large-brained animals. Here, we report the first use of these probes in humans, aiming to establish safe intraoperative use and assess feasibility for clinical and research applications. Nine patients undergoing neurosurgical procedures, including epilepsy or tumor resection and deep brain stimulation (DBS) implantation, were enrolled. Successful intraoperative recordings were obtained from surface and deep cortical structures without probe breakage or adverse events. Compared with conventional electrodes, the Neuropixels probe enabled dense sampling across multiple parenchymal depths with submillisecond temporal resolution. Recordings were obtained from deep targets including the hippocampus and cingulate cortex, as well as from regions that are challenging to access with single-unit precision, such as the superior frontal sulcus. Custom tools and refined workflows lowered technical barriers for operative use and improved recording stability. Neural activity was observed across all recordings. Neuropixels 1.0-NHP Long probes can be deployed in the human operating room, enabling simultaneous recordings from multiple brain structures at single-neuron resolution. These methods expand opportunities for studying human brain function and pathology in vivo, and may ultimately support the development of more precise neurosurgical interventions.

q-bio.NC↗

Making brain-machine interfaces robust to future neural variability

A major hurdle to clinical translation of brain-machine interfaces (BMIs) is that current decoders, which are trained from a small quantity of recent data, become ineffective when neural recording conditions subsequently change. We tested whether a decoder could be made more robust to future neural variability by training it to handle a variety of recording conditions sampled from months of previously collected data as well as synthetic training data perturbations. We developed a new multiplicative recurrent neural network BMI decoder that successfully learned a large variety of neural-to- kinematic mappings and became more robust with larger training datasets. When tested with a non-human primate preclinical BMI model, this decoder was robust under conditions that disabled a state-of-the-art Kalman filter based decoder. These results validate a new BMI strategy in which accumulated data history is effectively harnessed, and may facilitate reliable daily BMI use by reducing decoder retraining downtime.

q-bio.NC↗