Mouse Gestures Research Articles

Nose-Driven Cursor Control: An Assistive System for Disabled Individuals Using OCRM Similarity Tracking

This paper introduces a visual mouse system designed to assist disabled individuals in controlling computer mouse functions, such as cursor movement and clicking actions, using simple body gestures. Specifically, the system tracks the user’s nose tip to execute mouse commands, providing an untouchable alternative for desktop interaction. The main goal is to create an accessible, low-cost solution using widely available hardware to make computing more inclusive for individuals with physical disabilities. Using a camera, the system captures and translates user head movements into mouse gestures on the screen. The proposed methodology is built around visual tracking, isolating a sample region from each video frame and then identifying that region in subsequent frames through Optimal Cost Region Matching (OCRM). This similarity measure ensures accurate and seamless tracking of the user's movement, providing an effective control mechanism. Experimental results demonstrate that the OCRM-based visual mouse system outperforms existing Camera Mouse applications regarding reliability and efficiency. The proposed approach delivers a flexible, cost-effective, and consistent solution for enhancing computer accessibility for disabled users.

Gesture Controlled Virtual Mouse with Voice Assistant

The usage of hand gestures to operate digital devices has grown in recent years, and developing accurate gesture recognition systems using computer vision technology is becoming easier. Voice assistants have also become more common, with developments in artificial intelligence (AI) and natural language processing (NLP) allowing for more accurate voice recognition. Our goal is to create a system which combines both gesture-controlled mouse functionality and voice assistant to execute things that a regular mouse and keyboard can. The GCVA (Gesture Controlled Voice Assistant) comes with a gesture-controlled mouse and a voice assistant. A gesture-controlled mouse uses OpenCV and Media-Pipe, whereas a voice assistant uses AI and NLP concepts to recognize the voice commands given by the user. The system should be used by performing hand gestures and executing the right click, left click, drag, drop, volume control and computer cursor functions, thereby omitting the need for a physical mouse. The success of this system is observed in terms of practicality, stability, and compatibility with physical mechanisms. The voice assistant works and responds to pre-defined commands present in the system. The GCVA system was tested by four different users in order to generate the accuracy scores of mouse gestures and response time graphs for voice assistant. The system yields an accuracy rate of 98%. The system's accuracy and usability showed that hand gesture detection and voice assistants have the potential to replace traditional input devices in the future.

Rapid Setup of Tissue Microarray and Tiled Area Imaging on the Multiplexed Ion Beam Imaging Microscope using the Tile/SED/Array Interface.

Multiplexed ion beam imaging (MIBI) is a next-generation mass spectrometry-based microscopy techniquethat generates 40+ plex images of protein expression in histologic tissues, enabling detailed dissection of cellular phenotypes and histoarchitectural organization. A key bottleneck inoperation occurs when users select the physical locations on the tissue for imaging. As the scale and complexity of MIBI experiments have increased, the manufacturer-provided interface and third-party tools have become increasingly unwieldy for imaging large tissue microarrays and tiled tissue areas. Thus, a web-based, interactive, what-you-see-is-what-you-get (WYSIWYG) graphical interface layer - the tile/SED/array Interface (TSAI) - was developed for users to set imaging locations using familiar and intuitive mouse gestures such as drag-and-drop, click-and-drag, and polygon drawing. Written according to web standards already built into modern web browsers, it requires no installation of external programs, extensions, or compilers. Of interest to the hundreds of current MIBI users, this interface dramatically simplifies and accelerates the setup of large, complex MIBI runs.

Eye Controlled Mouse

Abstract: Numerous individuals who have neuro-locomotor deficits or are disabled by injury are unable to use PCs for abecedarian tasks, for example, sending or receiving dispatches, scouring the web, watching their TV programme, or stirring filmland. In a previous study, eyes were set up to be a great candidate for ubiquitous computing since they always move when they engage with calculating gear. Using this underlying knowledge from eye movements, these instances might be renewed using computers. For this aim, we offer a mouse gesture control mechanism that can only be used by mortal eyes. The goal of this study is to provide an open-source generic eye- gesture control system that can efficiently capture eye movements and allow the stoner to do conduct counterplotted to specified eye movements gestures via computer camera. After identifying it on the stoner's face, it tracks the pupil's motions. It must be accurate in real time for the stoner to use it comfortably, just like any other everyday prejudice.

Distance Semiotic Mediation through Orchestration Using a Video Conferencing System and Dynamic Geometry Software

In this study, we investigated the semiotic mediation process in an online class using both Zoom, a video conferencing system (VCS), and GeoGebra, a dynamic geometry software (DGS). We focused on the process by which students produced and shared their personal signs in various modalities and the instructional interventions in which the teacher used technologies to facilitate students’ semiotic activities. The findings show that teachers’ orchestration of a VCS and DGS support the distance mathematical communication of students through the production and sharing of multimodal semiotic resources. Students engaged in embodied interaction by sharing use not only verbal language but also non-verbal signs, such as dragging objects and mouse gestures. The collective production of multimodal semiotic bundles and the silent participation exclusively through gestures and diagramming without verbal utterances were also observed. Our case study provides examples of ways to tackle the rapidly changing mathematics education environment through distance semiotic mediation by orchestrating multiple technologies.

Authentication of a User Using a Combination of Hand Gesture and Online Signature

In real world most of the biometric system shows authentication of user using uni-modal system. In this they use single source of information to authenticate the user such as face segmentation approach or mouse gesture dynamic approach or hand gesture or online signature approach. Some of the limitations exposed by uni-model system can be overcome by using combination of any two approaches. We have already implemented authentication of a user using hand gesture and online signature separately. In order to increase the level of security it is very much essential to combine two authentication modes. The main goal of this paper is to develop a model to authenticate a user using a combination of hand gesture and online signature. When user logs in, they need to register by using both authentication methods. First user is asked to record a gesture using accelerometer followed by signature using digital tablet and pen. Hand gesture device consisting of a tri-axial accelerometer and microcontroller transmission to sense the hand gesture and collect the same. Online signature device consisting of a digital tablet, pen and microcontroller transmission board to sense the online gesture and collect the same. Personal computer receives both gesture and signature as a input and stores in database. During verification user is asked to give both gesture and signature for verification. This work ensures better accuracy and more effectiveness in result and also helps to minimize system error rate. As a result the performance of combination method is 100% and false acceptance ratio and false rejection ratio is zero. Our work has been found to provide better accuracy when compared to existing works.

Quantitative Analysis on the Interaction Fatigue of Natural Gestures

With the popularity of natural user interface (NUI), natural gesture interaction has become the mainstream. Using improper natural gestures for a long time will cause muscle fatigue, which leads to an increase in pathological problems such as tenosynovitis. To avoid the harm caused by improper gestures, this paper selects three daily interactive gestures as the research objects including browsing information, playing games and typing, and divides them into nine independent gestures. After denoising, filtering, segmenting, and extracting the parameters of the acquired surface electromyography (sEMG) signals, the time-domain, frequency-domain and time-frequency-domain are analyzed. The characteristics of the envelope waveform and power spectrum threshold, as well as the fatigue characteristics of nine independent gestures are obtained. The long short-term memory (LSTM), one of the recurrent neural network (RNN) methods is used to train nine independent gesture models. The fatigue characteristics of the integrated gestures are predicted by the trained LSTM series model. The energy consumption characteristics of integrated gestures in smartphones and PCs are obtained. It is found that the simple behavior of browsing in the integrated behavior is suitable for natural interactive gestures, and complex behaviors such as games or typing on PC have lower energy consumption than that of smartphones. Among the independent natural gestures, the energy consumption of clicking is higher than that of dragging. Compared with a behavior for the same purpose, the energy consumption of mouse gestures for PCs is much lower than that of smartphones. This study of gesture fatigue provides a theoretical basis for the design of gestures and the development of internet user products.

Seeking common ground while reserving differences in gesture elicitation studies

Gesture elicitation studies have been frequently conducted in recent years for gesture design. However, most elicitation studies adopted the frequency ratio approach to assign top gestures derived from end-users to the corresponding target tasks, which may cause the results get caught in local minima, i.e., the gestures discovered in an elicitation study are not the best ones. In this paper, we propose a novel approach of seeking common ground while reserving differences in gesture elicitation research. To verify this point, we conducted a four-stage case study on the derivation of a user-defined mouse gesture vocabulary for web navigation and then provide new empirical evidences on our proposed method, including 1) gesture disagreement is a serious problem in elicitation studies, e.g., the chance for participants to produce the same mouse gesture for a given target task without any restriction is very low, below 0.26 on average; 2) offering a set of gesture candidates can improve consistency; and 3) benefited from the hindsight effect, some unique but highly teachable gestures produced in the elicitation study may also have a chance to be chosen as the top gestures. Finally, we discuss how these findings can be applied to inform all gesture-based interaction design.

Finding and Emulating Keyboard, Mouse, and Touch Interactions and Gestures while Crawling RIAs

Existing techniques for crawling Javascript-heavy Rich Internet Applications tend to ignore user interactions beyond mouse clicking, and therefore often fail to consider potential mouse, keyboard and touch interactions. We propose a new technique for automatically finding and exercising such interactions by analyzing and exercising event handlers registered in the DOM. A basic form of gesture emulation is employed to find states accessible via swiping and tapping. Testing the tool against 6 well-known gesture libraries and 5 actual RIAs, we find that the technique discovers many states and transitions resulting from such interactions, and could be useful for cases such as automatic test generation and error discovery, especially for mobile web applications.

Repertoire Remix: Integrating Musical Preferences from Remote Audience in a Live Networked Improvisation

Repertoire Remix enables remote audience members to dynamically suggest their musical preferences for live web-streaming musical improvisation sessions. The semantic web interface encourages remote participants to collaboratively use “stirring” mouse gestures to influence the size of graphical bubbles that contain composers’ names. The accumulated weight is then interpreted by musicians to improvise. This paper documents the first pilot run of the Repertoire Remix system, explores challenges in designing a real-time shared music-style arranging system for networked live improvisation, and interprets the resulting performance by assessing the participants’ mouse gestures collected during the pilot run.

PopPlanner: visually constructing demographic models for simulation.

Currently there are a number of coalescent simulation programs that support a wide range of features, such as arbitrary demographic models, migration, and sub structure. Defining the model is done typically with either text files or command line switches. Although this has proven to be a powerful method of defining models of high complexity, it is often error prone and difficult to read without familiarity both with command lines and the program in question. A intuitive GUI based population structure program that can both read and write applicable command lines would dramatically simplify the construction, modification, and error checking of such models by a wider user base.Results: PopPlanner is a tool to both construct and inspect complicated demographic models visually with a GUI where the user's primary interaction is through mouse gestures. Because of their popularity, we focus on ms and by extension msms, command line coalescent simulation programs. Our program can be used to find errors with existing command lines, or to build original command lines. Furthermore, the graphical output supports a number of editing and output features including export of publication quality figures.

4-D Authentication System Using Mouse Gesture

Computer technology is reaching new milestones with every passing day but authentication schemes are still weak in their approach. Many cases of forgeries, hacked accounts are coming up every day. Many authentication schemes have been proposed but each has some drawbacks. Hence, the 3D authentication scheme has been introduced. The 3D password is a multi-layer, multifactor authentication mechanism. It consists of a 3D virtual environment on which a user has to perform certain actions. The sequence of user interactions determines the user’s 3D password. It combines all existing authentication schemes like textual, graphical and bio-metrics into a single 3D virtual environment. This report presents a study of the 3D password and an approach to strengthen it by way of adding a Fourth dimension (4D), that deals with gesture recognition and time recording, and that would help strengthen the authentication paradigm altogether. Hence we attempt to propose a 4D password as a super class of 3D password.

Biometric Validation by Storing different Patterns using Mouse Gesture Signatures

Biometric Validation by Storing different Patterns using Mouse Gesture Signatures

Scalable methods to integrate task knowledge with the Three-Weight Algorithm for hybrid cognitive processing via optimization

Abstract In this paper we consider optimization as an approach for quickly and flexibly developing hybrid cognitive capabilities that are efficient, scalable, and can exploit task knowledge to improve solution speed and quality. Given this context, we focus on the Three-Weight Algorithm, which is interruptible, scalable, and aims to solve general optimization problems. We propose novel methods by which to integrate diverse forms of task knowledge with this algorithm in order to improve expressiveness, efficiency, and scaling across a variety of problems. To demonstrate these techniques, we focus on two large-scale constraint-satisfaction domains, Sudoku and circle packing. In Sudoku, we efficiently and dynamically integrate knowledge of logically deduced sub-problem solutions; this integration leads to improved system reactivity and greatly reduced solution time for large problem instances. In circle packing, we efficiently integrate knowledge of task dynamics, as well as real-time human guidance via mouse gestures; these integrations lead to greatly improved system reactivity, as well as world-record-breaking solutions on very large packing problems. These results exemplify how cognitive architecture can integrate high-level knowledge with powerful optimization techniques in order to effectively and efficiently contend with a variety of cognitive tasks.

GRAPHICAL AUTHENTICATION USING REGION BASED GRAPHICAL PASSWORD

Password authentication is failing as an authentication since it increases the user burden to remember the passwords. Graphical authentication is proposed as a alternative for textual passwords since it may be easy for users to remember.In this paper we propose a new image region selection based graphical password scheme. We are going to present a new technique for authentication which is based on the tracking of mouse motions on an image called mouse gestures for selecting regions in the image. In general, a gesture is a sequence of interactions with the application, which represents one of the Specified symbols. A mouse gesture is a continuous, directed sequence of the mouse cursor movements with the clearly distinguished start and end. A set of gestures may be stored in a database called gesture classes for each user. Users are allowed to select a set of random images and a gesture for each image. Some tolerance level is also given for each gesture. When logging in if the user draws the correct gesture using mouse the user will be treated as an authenticated user. Mouse gestures are captured through bounding box and corner detection algorithms. This method provides more security than cued click points where the user is allowed to click on a particular point called pass point for authentication which is more vulnerable to hackers.

심성모형 기반의 마우스 제스처 개발

Various web browsers offer mouse gesture functions because they are convenient input methods. Mouse gestures enable users to move to the previous page or tab without clicking its relevant icon or menu of the web browser. To maximize the efficiency of mouse gestures, they should be designed to match users` mental models. Mental models of human beings are used to make accurate predictions and reactions when certain information has been recognized by humans. This means providing users with appropriate information about mental models will lead to fast understanding and response. A cognitive response test was performed in order to evaluate whether the mouse gestures easily associate with their respective functional meanings or not. After extracting mouse gestures which needed improvement, those were redesigned to reduce cognitive load via sketch maps. The methods presented in this study will be of help for evaluating and designing mouse gestures.

Biometric Authentication Using Mouse Gesture Dynamics

The mouse dynamics biometric is a behavioral biometric technology that extracts and analyzes the movement characteristics of the mouse input device when a computer user interacts with a graphical user interface for identification purposes. Most of the existing studies on mouse dynamics analysis have targeted primarily continuous authentication or user reauthentication for which promising results have been achieved. Static authentication (at login time) using mouse dynamics, however, appears to face some challenges due to the limited amount of data that can reasonably be captured during such a process. In this paper, we present a new mouse dynamics analysis framework that uses mouse gesture dynamics for static authentication. The captured gestures are analyzed using a learning vector quantization neural network classifier. We conduct an experimental evaluation of our framework with 39 users, in which we achieve a false acceptance ratio of 5.26% and a false rejection ratio of 4.59% when four gestures were combined, with a test session length of 26.9 s. This is an improvement both in the accuracy and validation sample, compared to the existing mouse dynamics approaches that could be considered adequate for static authentication. Furthermore, to our knowledge, our work is the first to present a relatively accurate static authentication scheme based on mouse gesture dynamics.

Keyframe-based Learning from Demonstration

We present a framework for learning skills from novel types of demonstrations that have been shown to be desirable from a Human–Robot Interaction perspective. Our approach—Keyframe-based Learning from Demonstration (KLfD)—takes demonstrations that consist of keyframes; a sparse set of points in the state space that produces the intended skill when visited in sequence. The conventional type of trajectory demonstrations or a hybrid of the two are also handled by KLfD through a conversion to keyframes. Our method produces a skill model that consists of an ordered set of keyframe clusters, which we call Sequential Pose Distributions (SPD). The skill is reproduced by splining between clusters. We present results from two domains: mouse gestures in 2D and scooping, pouring and placing skills on a humanoid robot. KLfD has performance similar to existing LfD techniques when applied to conventional trajectory demonstrations. Additionally, we demonstrate that KLfD may be preferable when demonstration type is suited for the skill.

Identifying Aggregate Scanning Strategies to Improve Usability Evaluations

Automated analysis methods are needed to convert eye tracking data into meaningful descriptions of high-level scanning strategies that can help usability professionals understand how designs are viewed. Prior techniques for analyzing eye tracking sequences, or scanpaths, are limited in their ability to find aggregated visual search strategies for a group of observers. We describe a pattern analysis tool that finds clusters of sequentially matching patterns among multiple scanpaths. Scanning strategies are clustered hierarchically, then represented as aggregate scanpaths. The tool can also match scanpaths against a hypothetical scanning strategy, input by mouse gesture. Use of the pattern analysis tool was demonstrated using a 114-participant eye tracking dataset in which several aggregate scanning strategies were identified. Matching against a hypothetical, counter-clockwise scanning strategy was also included in the demonstration. The analysis tool provides a valuable resource for aggregating or grouping scanning strategies, a vital step toward generalizing eye tracking results to guide design recommendations for improving usability.

The influence of violent and nonviolent computer games on implicit measures of aggressiveness

We examined the causal relationship between playing violent video games and increases in aggressiveness by using implicit measures of aggressiveness, which have become important for accurately predicting impulsive behavioral tendencies. Ninety-six adults were randomly assigned to play one of three versions of a computer game that differed only with regard to game content (violent, peaceful, or abstract game), or to work on a reading task. In the games the environmental context, mouse gestures, and physiological arousal-as indicated by heart rate and skin conductance-were kept constant. In the violent game soldiers had to be shot, in the peaceful game sunflowers had to be watered, and the abstract game simply required clicking colored triangles. Five minutes of play did not alter trait aggressiveness, yet an Implicit Association Test detected a change in implicit aggressive self-concept. Playing a violent game produced a significant increase in implicit aggressive self-concept relative to playing a peaceful game. The well-controlled study closes a gap in the research on the causality of the link between violence exposure in computer games and aggressiveness with specific regard to implicit measures. We discuss the significance of importing recent social-cognitive theory into aggression research and stress the need for further development of aggression-related implicit measures.