Automatic Speech Activity Recognition from MEG Signals Using Seq2Seq Learning

Date

2019-03

ORCID

Journal Title

Journal ISSN

Volume Title

Publisher

IEEE Computer Society

item.page.doi

Abstract

Accurate interpretation of speech activity from brain signals is critical for understanding the relationship between neural patterns and speech production. Current research on speech activity recognition from the brain activity heavily relies on the region of interest (ROI) based functional connectivity analysis or source separation strategies to map the activity as a spatial localization of a brain function. Albeit effective, these methods require prior knowledge of the brain and expensive computational effort. In this study, we investigated automatic speech activity recognition from brain signals using machine learning. Neural signals of four subjects during four stages of a speech task (i.e., rest, perception, preparation, and production) were recorded using magnetoencephalography (MEG), which has an excellent temporal and spatial resolution. First, a deep neural network (DNN) was used to classify the four whole tasks from the MEG signals. Further, we trained a sequence to sequence (Seq2Seq) long short-term memory-recurrent neural network (LSTM-RNN) for continuous (sample by sample) prediction of the speech stages/tasks by leveraging its sequential pattern learning paradigm. Experimental results indicate the effectiveness of both DNN and LSTM-RNN for automatic speech activity recognition from MEG signals. © 2019 IEEE.

Description

Due to copyright restrictions and/or publisher's policy full text access from Treasures at UT Dallas is limited to current UTD affiliates (use the provided Link to Article).

Keywords

Brain, Brain mapping, Neural networks (Neurobiology), Image segmentation, Long-term memory, Magnetoencephalography, Speech, Automatic speech recognition, Speech perception, Short-term memory

item.page.sponsorship

National Institutes of Health (NIH) under award number R03 DC013990

Rights

©2019 IEEE

Citation