Gesture Controlled Mouse and Voice Assistant

Abstract

Abstract: This project advocates a Human Computer Interaction method where cursor movement can be controlled via real-time camera by making use of human hand postures recognition. This method is an alternative to current techniques, which include manually pressing buttons or using physical computer mouse. Instead, it uses camera and computer vision software to manage different mouse events and can carry out any action that a conventional computer mouse can. The Virtual Mouse color recognition application will continuously gather real-time photos, which will then go through several conversions and filters. Once everything is converted, the application will use image processing to extract coordinates of specified color position from converted frames. The process then compares current color schemes within the frames to a list of color combinations, where each combination corresponds to a particular set of mouse operations. If the current color scheme matches, the application will perform the mouse command, which will be converted into a real mouse command on the user's computer. In addition to that authors have also developed a voice assistant to improve user productivity by managing routine tasks of the user and by providing information from online sources to the user because voice assistant is effortless to use.