Gridlines approach for dynamic analysis in speech ultrasound data: A multimodal app

Simon Gonzalez

doi:10.16995/labphon.6463

Abstract

Having access to efficient technologies is essential for the accurate description and analysis of articulatory speech patterns. In the area of tongue ultrasound studies, the visualization/analysis processes generally require a solid knowledge of programming languages as well as a deep understanding of articulatory phenomena. This demands the use of a variety of programs for an efficient use of the data collected. In this paper we introduce a multimodal app for visualizing and analyzing tongue contours: UVA—Ultrasound Visualization and Analysis. This app combines the computational power of R and the interactivity of Shiny web apps to allow users to manipulate and explore tongue ultrasound data using cutting-edge methods. One of the greatest strengths of the app is that it has the capability of being modified to adapt to users’&nbsp;needs. This has potential as an innovative tool for diverse academic and industry audiences.

Highlights

Speech analysis technologies have played a fundamental role in the understanding of how human language uses all available articulators for communication
The app implements a wide range of visualization parameters and carries out analysis of speech ultrasound data for both static and dynamic approaches
In relation to the different stages of the workflow explained above, the app aims to enhance and facilitate mainly stages four (Visualization) and five (Analysis) with a tagging functionality for stage three (Data Wrangling). Since it does not offer collection and contour extraction, users are assumed to have tongue contours extracted in xy Cartesian coordinates, either in pixels and/or millimetres

Summary

Introduction

Speech analysis technologies have played a fundamental role in the understanding of how human language uses all available articulators for communication. It is important to note that there are standalone programs which can efficiently do ultrasound analysis and incorporate different stages simultaneously, for example the Articulate Assistant Advanced (AAA) software (Wrench, 2012), which is powerful and of great use These programs generally require specialized equipment and they do not tend to be available for code expansion/modification based on users’ needs. The app implements a wide range of visualization parameters and carries out analysis of speech ultrasound data for both static and dynamic approaches. In relation to the different stages of the workflow explained above, the app aims to enhance and facilitate mainly stages four (Visualization) and five (Analysis) with a tagging functionality for stage three (Data Wrangling) Since it does not offer collection and contour extraction, users are assumed to have tongue contours extracted in xy Cartesian coordinates, either in pixels and/or millimetres. This is relevant when some data is being considered for deletion; having the option to visualize it beforehand is very important

Visualization The visualization section has three parts

Conclusions