Development of a Telerobotic System to Assist the Physically Challenged Using Non-Contact Vision-Based Sensing and Command

Abstract

It is often a problem for a physically challenged person to perform a simple routine task such as eating, moving around, and picking up things on a shelf. Usually, these tasks require assistance from a capable person. However, this total or partial reliance on others for daily routines may be bothersome to the physically challenged and diminishes their self-esteem. Moreover, getting around in a wheel chair, for example, requires the use of some form of a joystick, which is usually not so user-friendly. At Texas A&M Qatar, we developed two vision-based motion detection and actuation systems, which can be used to control the motion of a wheel chair without the use of a joystick and remotely control the motion of a service robot for assistance with daily routines. The first vision system detects the orientation of the face whereas the second detects the motion of color tags placed on the person's body. The orientation of the face and the motion of the color tags are detected using a CCD camera and could be used to command the wheel chair and the remote robot wirelessly. The computation of the color tags’ motion is achieved through image processing using eigenvectors and color system morphology. Through inverse dynamics and coordinate transformation, the motion of the operator's head, limbs, and face orientation could be computed and converted to the appropriate motor angles on the wheelchair and the service robot. Our initial results showed that it takes, on average, 65 milliseconds per calculation. The systems performed well even in complex environments with errors that did not exceed 2 pixels with a response time of about 0.1 seconds. The results of the experiments are available at: http://www.youtube.com/watch?v=5TC0jqlRe1U, http://www.youtube.com/watch?v=3sJvjXYgwVo, http://www.youtube.com/watch?v=yFxLaVWE3f8, and http://www.youtube.com/watch?v=yFxLaVWE3f8. It is our intent to implement the vision based sensing system on an actual wheelchair and service robot and test it using a physically challenged person.