Augmented Virtual Mouse System with Enhanced Gesture Recognition

Abstract

This paper presents the development of a contactless system designed to serve as an alternative to the traditional physical input device, such as a computer mouse or touchpad, utilized in human-computer interaction. The proposed system aims to offer a more convenient and hygienic way of interacting with computer devices by using hand gestures instead of physical contact with the mouse system or machine interface. The system has potential applications in various fields including healthcare, public interfaces, and gaming. Existing gesture-controlled input systems lack a comprehensive touchless and accurate system capable of performing almost every operation of a physical mouse. The proposed approach involves recognising the hand and its gestures. Hand tracking and dynamic gesture recognition are incorporated to ensure a smooth and hassle-free user experience during interaction. The efficient Mediapipe framework is used to detect and track hand movements from video frames captured through a computing device Camera.Computer vision algorithms are employed for gesture recognition within each frame, along with Python libraries for processing gestures. Sufficient gestures are added into the system, enabling it to execute a wide range of mouse functions with substantial precision metrics. This paper provides details regarding the development and overall performance of the system. Key Words: hands, mouse, contactless, gesture, OpenCV, mediapipe