Dynamic component analysis for multi-input/multi-output problems, with application to speech and neurophysiology

Abstract

We explore a set of methods referred to collectively as dynamic component analysis (DCA) to derive dictionaries of dynamical patterns for speech (TIMIT database). The methods use spatio-temporal singular value decomposition (ST-SVD) and common spatio-temporal pattern (CSTP) matrix computations. When used on the speech spectrogram, these yield a transformation to a new set of time series representing extracted features at reduced bandwidth. The method is computationally efficient (closed-form solutions suitable for real-time) and robust to additive noise, with diverse applications in speech processing and general multi-input/multi-output (MI/MO) modeling. When used to predict a single neural output (MI/SO), this gives an efficient new method for deriving the spectro-temporal receptive field (STRF), which is shown in our human cortical data to yield improved predictions. We also use DCA to reconstruct speech from cortical activity, wherein dynamical dictionaries for electrocortical data are derived, with application to brain-computer interfaces (BCI).