Using machine learning to identify articulatory gestures in time course data

Abstract

One difficulty in working with articulatory data is objectively identifying phonological gestures, that is, distinguishing targeted gestural movement from general variability. Although human annotators are generally used, an automated approach to identifying meaningful patterns offers advantages in speed, consistency, and objective characterization of gestures (cf. Shaw and Kawahara 2017). This study examines Electromagnetic Articulography (EMA) data from seven American English speakers, aiming to identify and characterize pause postures (specific vocal tract configurations at prosodic boundaries; Katsika et al. 2014). Supervised machine learning using kernelized Support Vector Machine Classifiers (SVMs) took as training data 852 trajectories from three speakers analyzed to date, containing 104 pause postures identified by a human annotator, and classified the between-words lip aperture (LA) trajectory to identify tokens containing the pause posture, while also providing token-by-token gesture probability. Features of the curvature were extracted using Principal Component Analysis (PCA) and Discrete Cosine Transform (DCT) of both the actual LA trajectory and of the deviation from a direct word-to-word interpolation. The SVM achieves 94.0% classification accuracy in cross-validation tests, with Cohen’s Kappa showing machine-to-annotator agreement of 0.978. These methods of machine-learning-based curve classification are potentially useful and applicable to any time-course articulatory data. [Work supported by NIH and NSF.]