A Direct3D-Based Multi-model E-Learning System for Chinese Sign Language

Abstract

This paper describes a Direct3D-based multi-model e-learning system for Chinese sign language (CSL), which we design for both hearing-impaired people and hearing people to learn standard CSL. This system includes two key techniques: 3D human model animation and gesture motion retargeting. A novel virtual human animation approach is proposed to realize high-quality and realistic 3D human model animation. This approach utilizes skinned mesh technique based on Direct3D and the effect of 3D human model animation shows that the approach we proposed have good performance and can successfully solve the serration problem which is a definite flaw of traditional CSL. In addition, a constraint-based Inverse Kinematics (IK) algorithm for gesture motion retargeting is also presented in this paper. As there are multiple models in our system, the proposed algorithm is aimed to retarget the sign language data captured by motion capture device to different 3D human models with different sizes and proportions. Experimental results prove that it can greatly improve the exactness of sign language.