Expert Gestures Research Articles

Gesture recognition and sensorimotor learning‐by‐doing of motor skills in manual professions: A case study in the wheel‐throwing art of pottery

AbstractThis paper presents a methodological framework for the use of gesture recognition technologies in the learning/mastery of the gestural skills required in wheel‐throwing pottery. In the case of self‐instruction or training, learners face difficulties due to the absence of the teacher/expert and the consequent lack of guidance. Motion capture technologies, machine learning, and gesture recognition may provide a way of overcoming such issues. The proposed methodology is used to record and model expert gestures and then to compare this model in real time with the gestures performed by the learner. Differences in kinematic aspects such as hand distances are detected, and optical/sonic sensorimotor feedback is provided to the learner by the system, alerting him/her when errors occur and guiding him/her to achieve better results. In the case described here, the system was evaluated with 11 learners. With the use of our system, the gestural performance of learners during self‐training has been improved in comparison to cases of self‐training without computer assistance.

Eluding the Physical Constraints in a Nonlinear Interaction Sound Synthesis Model for Gesture Guidance

In this paper, a flexible control strategy for a synthesis model dedicated to nonlinear friction phenomena is proposed. This model enables to synthesize different types of sound sources, such as creaky doors, singing glasses, squeaking wet plates or bowed strings. Based on the perceptual stance that a sound is perceived as the result of an action on an object we propose a genuine source/filter synthesis approach that enables to elude physical constraints induced by the coupling between the interacting objects. This approach makes it possible to independently control and freely combine the action and the object. Different implementations and applications related to computer animation, gesture learning for rehabilitation and expert gestures are presented at the end of this paper.

Modelling Gestural Know-how in Pottery Based on State-space Estimation and System Dynamic Simulation

This paper presents a methodology for modelling expert gestural performances in wheel-throwing pottery. The approach is based on building an operational model that describes how expert gestures are performed, taking also into consideration relationships between different parts of the body. This model is estimated using state-space estimation methodology and its predictive performance is evaluated using system dynamic simulation. Moreover, the confidence bounds derived of the expert gesture performance in wheel-throw pottery are computed. They could be used as benchmarks in real-time experiments in order to generate the appropriate feedback for sensorimotor learning of gestures.

Music Gestural Skills Development Engaging Teachers, Learners and Expert Performers

This article presents a platform for learning theoretical knowledge and practical motor skills of musical gestures by combining functionalities of Learning Management Systems (LMS) and Serious Gaming (SG). The teacher designs his/her educational scenario that can be articulated by both theoretical and practical activities. The learner accesses online multimedia courses by using his/her LMS client which can be a computer, tablet or smartphone and the serious game by using his/her computer and the motion capture sensors. During practicing, his/her gestures are compared in real-time with the expert gestures and s/he is evaluated both in terms of correct fingerings and kinematics. Finally, the platform offers a single profile for the learner for theoretical and practical activities.

Optimal synthesis of a spherical parallel mechanism for medical application

SUMMARYThis paper introduces the design and the optimization of a probe holder robot for tele-echography applications. To define its kinematic architecture, an approach based on motion capture of an expert's gestures during ultrasound examinations was proposed. The medical gestures analyzed consisted of ultrasound probe movements and were used to characterize the kinematic specifications of the proposed manipulator. The selected architecture was a Spherical Parallel Mechanism (SPM) with 3 degrees of freedom (DoF) and its optimal synthesis was performed using real-coded Genetic Algorithms (GA). The optimization criteria and constraints were established thanks to the collaboration of medical experts and were successively formulated and solved using mono-objective and multi-objective functions.

A tele-operated mobile ultrasound scanner using a light-weight robot

This paper presents a new tele-operated robotic chain for real-time ultrasound image acquisition and medical diagnosis. This system has been developed in the frame of the Mobile Tele-Echography Using an Ultralight Robot European Project. A light-weight six degrees-of-freedom serial robot, with a remote center of motion, has been specially designed for this application. It holds and moves a real probe on a distant patient according to the expert gesture and permits an image acquisition using a standard ultrasound device. The combination of mechanical structure choice for the robot and dedicated control law, particularly nearby the singular configuration allows a good path following and a robotized gesture accuracy. The choice of compression techniques for image transmission enables a compromise between flow and quality. These combined approaches, for robotics and image processing, enable the medical specialist to better control the remote ultrasound probe holder system and to receive stable and good quality ultrasound images to make a diagnosis via any type of communication link from terrestrial to satellite. Clinical tests have been performed since April 2003. They used both satellite or Integrated Services Digital Network lines with a theoretical bandwidth of 384 Kb/s. They showed the tele-echography system helped to identify 66% of lesions and 83% of symptomatic pathologies.