A camera based human computer interaction through virtual keyboard assistant

Abstract

Human Computer Interaction has evolved with time and exploited almost all the possible relationships that human can have with computers. The present scenario of using the computer is often inaccessible to people with severe motor impairment, which cannot inhabit the traditional keyboard and mouse, and require another specialized input device. A Camera mouse Human-Computer Interface (HCI) based virtual assistant is designed and planned to be evaluated with 15 subjects and hoped to work for individuals with advanced Amyotrophic Lateral Sclerosis (ALS), all of whom were given tasks to execute through the virtual assistant. In the proposed system, this includes primarily, a virtual keyboard with an embedded Internet browser. The virtual keyboard allows users with movement impairments, basically hand movement issues, to directly interact with the computer. The keyboard has varied functions, apart from just typing; namely, searching anything on google, login to social media and checking emails, text to speech facilities and has a database that holds all the basic helper commands for the person. The camera mouse is software that allows people to interact with the computer by registering any part of their body and then using that part as the pointing device to work on the computer system. The participants of the evaluation will be both, subjects that have knowledge about the HCI interface and those completely new to it. We define additional criteria for unrestricted internet access for evaluation of the presented and future internet browsers, Camera based browser provides unrestricted access and enables free web surfing for individuals with motor impairments. This system improves the subject’s quality of life and provides uninterrupted access for the subject to use the computer.