A Novel Approach for Allocating Mathematical Expressions to Visual Speech Signals

Mohammad Hossein Sadaghiani ,Dino Isa,Niusha Shafiabady

doi:10.1177/2158244015611937

Mohammad Hossein Sadaghiani , Dino Isa + Show 1 more

Open Access

https://doi.org/10.1177/2158244015611937

Copy DOI

Journal: SAGE Open	Publication Date: Oct 1, 2015
Citations: 3	License type: cc-by

Affiliation: University of Nottingham Malaysia Campus

Abstract

In this article, visual speech information modeling analysis by explicit mathematical expressions coupled with words’ phonemic structure is presented. The visual information is obtained from deformation of lips’ dimensions during articulation of a set of words that is called visual speech sample set. The continuous interpretation of the lips’ movement has been provided using Barycentric Lagrange Interpolation producing a unique mathematical expression named visual speech signal. Hierarchical analysis of the phoneme sequences has been applied for words’ categorization to organize the database properly. The visual samples were extracted from three visual feature points chosen on the lips via an experiment in which two individuals pronounced the aforementioned words. The simulation results show that each individual word can be represented by a mathematical expression or visual speech signal whereas the sample sets can also be derived from the same mathematical expression, and this is a significant improvement over the popular statistical methods.

Highlights

The audiovisual speech recognition and visual speech synthesizers are two interfaces for human and machine interaction (Chin, Seng, & Ang, 2012)
Selecting appropriate phoneme–viseme mapping table for analysis of phonemic structure would be beneficial in deriving the practical visual speech signal expressions that are adaptable to audiovisual speech recognition systems
The reason for using Barycentric Lagrange interpolation (BLI) is the ability of formulating the visual speech signals which involve the sample sets extracted from the visual feature points without Rounge effect

Summary

Introduction

The audiovisual speech recognition and visual speech synthesizers are two interfaces for human and machine interaction (Chin, Seng, & Ang, 2012). In lip-reading systems, the dynamics of visual speech in image sequences are extracted by focusing on the appearance of the articulator organs like lips’ geometry. The focus of this article is on extracting the geometry of lips during articulation to suggest a mathematical model of lips’ dynamics as there is no specific attempt for expressing the visual speech data during articulation by an explicit mathematical formula.

Results

Conclusion