Abstract
An algorithm for processing the continuous stream of incoming video data to detect and recognize signed expressions is proposed. The aim is to improve an assistive technology system. The method is based on the dynamic flow of expression models between a repository of inactive models and pools of active and completed models. When modeling signed expressions, so-called visemes are used. These are units smaller than words - the equivalent of phonemes known from speech recognition. Visemes are extracted by clustering descriptors of the holistic skeletons. The sequential-simultaneous nature of gestures is considered, and the modeling is carried out in four parallel channels related to the position and shape of both hands. The conditions for activation, deactivation, and recognition of the model as completed are formulated, assuming asynchronous changes in individual channels. The method is successfully tested on a demanding dataset recorded by potential users.
Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
