AbstractThis paper develops a complete communication support system for speech and hearing disorders using lip reading. The proposed system is a highâlevel system and consists of a face detector using the ViolaâJones method, lip extraction based on the active appearance model, dynamic programming matching, automatic utterance section extraction, camera control, and message output. We implemented the system in a commercial laptop, and carried out the recognition experiments on 50 Japanese phrases with four subjects for more than 1 month and obtained high recognition accuracy. Furthermore, we demonstrated it in various situations such as the sitting position, supine position, and holding position. We confirmed that our system can be used in various situations. Š 2013 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.
Read full abstract