Automatic Speech Activity Recognition from MEG Signals Using Seq2Seq Learning




Journal Title

Journal ISSN

Volume Title


IEEE Computer Society


Accurate interpretation of speech activity from brain signals is critical for understanding the relationship between neural patterns and speech production. Current research on speech activity recognition from the brain activity heavily relies on the region of interest (ROI) based functional connectivity analysis or source separation strategies to map the activity as a spatial localization of a brain function. Albeit effective, these methods require prior knowledge of the brain and expensive computational effort. In this study, we investigated automatic speech activity recognition from brain signals using machine learning. Neural signals of four subjects during four stages of a speech task (i.e., rest, perception, preparation, and production) were recorded using magnetoencephalography (MEG), which has an excellent temporal and spatial resolution. First, a deep neural network (DNN) was used to classify the four whole tasks from the MEG signals. Further, we trained a sequence to sequence (Seq2Seq) long short-term memory-recurrent neural network (LSTM-RNN) for continuous (sample by sample) prediction of the speech stages/tasks by leveraging its sequential pattern learning paradigm. Experimental results indicate the effectiveness of both DNN and LSTM-RNN for automatic speech activity recognition from MEG signals. © 2019 IEEE.


Due to copyright restrictions and/or publisher's policy full text access from Treasures at UT Dallas is limited to current UTD affiliates (use the provided Link to Article).


Brain, Brain mapping, Neural networks (Neurobiology), Image segmentation, Long-term memory, Magnetoencephalography, Speech, Automatic speech recognition, Speech perception, Short-term memory

National Institutes of Health (NIH) under award number R03 DC013990


©2019 IEEE