arXiv subjects

Seo-Hyun Lee

Publications and source records attributed to Seo-Hyun Lee.

24 records · Page 2Linked to original sources

arXiv2023

Towards Voice Reconstruction from EEG during Imagined Speech

Translating imagined speech from human brain activity into voice is a challenging and absorbing research issue that can provide new means of human communication via brain signals. Endeavors toward reconstructing speech from brain activity have shown their potential using invasive measures of spoken speech data, however, have faced challenges in reconstructing imagined speech. In this paper, we propose NeuroTalk, which converts non-invasive brain signals of imagined speech into the user's own voice. Our model was trained with spoken speech EEG which was generalized to adapt to the domain of imagined speech, thus allowing natural correspondence between the imagined speech and the voice as a ground truth. In our framework, automatic speech recognition decoder contributed to decomposing the phonemes of generated speech, thereby displaying the potential of voice reconstruction from unseen words. Our results imply the potential of speech synthesis from human EEG signals, not only from spoken speech but also from the brain signals of imagined speech.

eess.AS↗

arXiv2022

Toward Imagined Speech based Smart Communication System: Potential Applications on Metaverse Conditions

Metaverse provides an alternative platform for human interaction in the virtual world. Since virtual platform holds few restrictions in changing the surrounding environments or the appearance of the avatars, it can serve as a platform that reflects human thoughts or even dreams at least in the metaverse world. When it is merged together with the current brain-computer interface (BCI) technology, which enables system control via brain signals, a new paradigm of human interaction through mind may be established in the metaverse conditions. Recent BCI systems are aiming to provide user-friendly and intuitive means of communication using brain signals. Imagined speech has become an alternative neuro-paradigm for communicative BCI since it relies directly on a person's speech production process, rather than using speech-unrelated neural activity as the means of communication. In this paper, we propose a brain-to-speech (BTS) system for real-world smart communication using brain signals. Also, we show a demonstration of imagined speech based smart home control through communication with a virtual assistant, which can be one of the future applications of brain-metaverse system. We performed pseudo-online analysis using imagined speech electroencephalography data of nine subjects to investigate the potential use of virtual BTS system in the real-world. Average accuracy of 46.54 % (chance level = 7.7 %) and 75.56 % (chance level = 50 %) was acquired in the thirteen-class and binary pseudo-online analysis, respectively. Our results support the potential of imagined speech based smart communication to be applied in the metaverse world.

cs.HC↗

arXiv2021

EEG-Transformer: Self-attention from Transformer Architecture for Decoding EEG of Imagined Speech

Transformers are groundbreaking architectures that have changed a flow of deep learning, and many high-performance models are developing based on transformer architectures. Transformers implemented only with attention with encoder-decoder structure following seq2seq without using RNN, but had better performance than RNN. Herein, we investigate the decoding technique for electroencephalography (EEG) composed of self-attention module from transformer architecture during imagined speech and overt speech. We performed classification of nine subjects using convolutional neural network based on EEGNet that captures temporal-spectral-spatial features from EEG of imagined speech and overt speech. Furthermore, we applied the self-attention module to decoding EEG to improve the performance and lower the number of parameters. Our results demonstrate the possibility of decoding brain activities of imagined speech and overt speech using attention modules. Also, only single channel EEG or ear-EEG can be used to decode the imagined speech for practical BCIs.

cs.HC↗

arXiv2021

Voice of Your Brain: Cognitive Representations of Imagined Speech,Overt Speech, and Speech Perception Based on EEG

Every people has their own voice, likewise, brain signals dis-play distinct neural representations for each individual. Al-though recent studies have revealed the robustness of speech-related paradigms for efficient brain-computer interface, the dis-tinction on their cognitive representations with practical usabil-ity still remains to be discovered. Herein, we investigate the dis-tinct brain patterns from electroencephalography (EEG) duringimagined speech, overt speech, and speech perception in termsof subject variations with its practical use of speaker identifica-tion from single channel EEG. We performed classification ofnine subjects using deep neural network that captures temporal-spectral-spatial features from EEG of imagined speech, overtspeech, and speech perception. Furthermore, we demonstratedthe underlying neural features of individual subjects while per-forming imagined speech by comparing the functional connec-tivity and the EEG envelope features. Our results demonstratethe possibility of subject identification from single channel EEGof imagined speech and overt speech. Also, the comparison ofthe three speech-related paradigms will provide valuable infor-mation for the practical use of speech-related brain signals inthe further studies.

cs.HC↗

arXiv2020

Functional Connectivity of Imagined Speech and Visual Imagery based on Spectral Dynamics

Recent advances in brain-computer interface technology have shown the potential of imagined speech and visual imagery as a robust paradigm for intuitive brain-computer interface communication. However, the internal dynamics of the two paradigms along with their intrinsic features haven't been revealed. In this paper, we investigated the functional connectivity of the two paradigms, considering various frequency ranges. The dataset of sixteen subjects performing thirteen-class imagined speech and visual imagery were used for the analysis. The phase-locking value of imagined speech and visual imagery was analyzed in seven cortical regions with four frequency ranges. We compared the functional connectivity of imagined speech and visual imagery with the resting state to investigate the brain alterations during the imagery. The phase-locking value in the whole brain region exhibited a significant decrease during both imagined speech and visual imagery. Broca and Wernicke's area along with the auditory cortex mainly exhibited a significant decrease in the imagined speech, and the prefrontal cortex and the auditory cortex have shown a significant decrease in the visual imagery paradigm. Further investigation on the brain connectivity along with the decoding performance of the two paradigms may play a crucial role as a performance predictor.

cs.HC↗

arXiv2020

Spatio-Temporal Dynamics of Visual Imagery for Intuitive Brain-Computer Interface

Visual imagery is an intuitive brain-computer interface paradigm, referring to the emergence of the visual scene. Despite its convenience, analysis of its intrinsic characteristics is limited. In this study, we demonstrate the effect of time interval and channel selection that affects the decoding performance of the multi-class visual imagery. We divided the epoch into time intervals of 0-1 s and 1-2 s and performed six-class classification in three different brain regions: whole brain, visual cortex, and prefrontal cortex. In the time interval, 0-1 s group showed 24.2 % of average classification accuracy, which was significantly higher than the 1-2 s group in the prefrontal cortex. In the three different regions, the classification accuracy of the prefrontal cortex showed significantly higher performance than the visual cortex in 0-1 s interval group, implying the cognitive arousal during the visual imagery. This finding would provide crucial information in improving the decoding performance.

cs.HC↗