Finger Tracking System Research Articles

SHREC 2022 track on online detection of heterogeneous gestures

This paper presents the outcomes of a contest organized to evaluate methods for the online recognition of heterogeneous gestures from sequences of 3D hand poses. The task is the detection of gestures belonging to a dictionary of 16 classes characterized by different pose and motion features. The dataset features continuous sequences of hand tracking data where the gestures are interleaved with non-significant motions. The data have been captured using the Hololens 2 finger tracking system in a realistic use-case of mixed reality interaction. The evaluation is based not only on the detection performances but also on the latency and the false positives, making it possible to understand the feasibility of practical interaction tools based on the algorithms proposed. The outcomes of the contest’s evaluation demonstrate the necessity of further research to reduce recognition errors, while the computational cost of the algorithms proposed is sufficiently low.

FINGER COUNTING AND VIRTUAL MOUSE USING CVLEARN

The finger tracking system is focused on user- data interaction, where the user interacts with virtual data, by handling through the fingers the volumetric of a 3D object that we want to represent. The finger tracking system focuses on user-data interaction, where the user interacts with virtual data, by handling through the fingers the volumetric 3D object that we want to represent. The human interaction problem gave rise to the system.The main goal is to establish a communication between them and for them to be more intuitive, Finger tracking was developed. The system is capable of tracking the real-time positions in 2D and 3D orientations of the fingers and these intuitive hand movements are used for interactions. Our project introduces an approach where with mouse cursor movements can be controlled using the real- time camera. It is basically an alternative for the all types of mouses that we use. It uses a camera and computer vision technology to controls various mouse actions and is capable of performing every task that the physical computer mouse can.

Unleashing the Power of Active Sonar

The authors explore various applications that can be enabled by transforming mobile devices into active sonar systems. Specifically, they show how to use microphones and speakers available on existing smartphones to monitor minute body movements such as breathing. Using this, they build systems that will demonstrate the feasibility of achieving contactless solutions for sleep apnea diagnosis. Building on their active sonar approach, they also demonstrate the feasibility of subcentimeter finger tracking for around-device interaction using existing smartphones with no additional hardware. The resulting finger tracking system breaks a fundamental tension between the shrinking sizes of wearable and mobile devices and our ability to interact with them and achieves this without the need for additional hardware.

Integrated image monitoring system using head-mounted display for gasless single-port clampless partial nephrectomy.

A novel head-mounted display (HMD) offers a higher quality of endoscopic imagery in front of the eyes regardless of head position. We present an application of the HMD system as a personal integrated multi-image monitoring system in gasless single-port clampless partial nephrectomy (PN). Our HMD system displayed multiple forms of information as integrated, sharp, high-contrast images both seamlessly and synchronously using a four-split screen. The surgeon wearing an HMD display could continuously and simultaneously monitor the endoscopic, three-dimensional (3D) video and intraoperative ultrasound images. In addition, the operator can rotate the 3D video image using fingertip movements on the finger tracking system. All two clampless partial nephrectomies were safely completed within the operative time, blood loss was within usual limits and there were no complications. The integrated image HMD system might facilitate maneuverability and safety in minimally invasive clampless PN.

Instructive head-mounted display system: pointing device using a vision-based finger tracking technique applied to surgical education.

IntroductionThe head-mounted display (HMD) system is a novel personalized imaging monitoring system for use in a medical setting.AimTo support interactive intraoperative communication among HMD wearers, we integrated vision-based finger tracking into our system as a pointing device.Material and methodsOur vision-based finger tracking system is composed of a commercially available real-time video camera, which is mounted on the modern HMD, and computer software that enables tracking of the tip of the operator's index finger and superimposing the marker on the endoscopic view.ResultsWe used this system in an experimental demonstration. The operator used the finger-tracking pointer to explain the intraoperative findings of transurethral resection for bladder cancer to medical students.ConclusionsThis finger tracking system-based pointing device can function as a supportive tool for the HMD system, enabling interactive instruction and communication between the operator and other attending physicians or medical students.

A system for tracking braille readers using a Wii Remote and a refreshable braille display

This article describes a cheap and easy-to-use finger-tracking system for studying braille reading. It provides improved spatial and temporal resolution over the current available solutions and can be used with either a refreshable braille display or braille-embossed paper. In conjunction with a refreshable braille display, the tracking system has the unique capacity to implement display-change paradigms derived from sighted reading research. This will allow researchers to probe skilled braille reading in significantly more depth than has heretofore been possible.

Vision-Based Finger Detection, Tracking, and Event Identification Techniques for Multi-Touch Sensing and Display Systems

This study presents efficient vision-based finger detection, tracking, and event identification techniques and a low-cost hardware framework for multi-touch sensing and display applications. The proposed approach uses a fast bright-blob segmentation process based on automatic multilevel histogram thresholding to extract the pixels of touch blobs obtained from scattered infrared lights captured by a video camera. The advantage of this automatic multilevel thresholding approach is its robustness and adaptability when dealing with various ambient lighting conditions and spurious infrared noises. To extract the connected components of these touch blobs, a connected-component analysis procedure is applied to the bright pixels acquired by the previous stage. After extracting the touch blobs from each of the captured image frames, a blob tracking and event recognition process analyzes the spatial and temporal information of these touch blobs from consecutive frames to determine the possible touch events and actions performed by users. This process also refines the detection results and corrects for errors and occlusions caused by noise and errors during the blob extraction process. The proposed blob tracking and touch event recognition process includes two phases. First, the phase of blob tracking associates the motion correspondence of blobs in succeeding frames by analyzing their spatial and temporal features. The touch event recognition process can identify meaningful touch events based on the motion information of touch blobs, such as finger moving, rotating, pressing, hovering, and clicking actions. Experimental results demonstrate that the proposed vision-based finger detection, tracking, and event identification system is feasible and effective for multi-touch sensing applications in various operational environments and conditions.

Enlightened: the art of finger reading*

Abstract. For some forty years the eye‐tracking technology has facilitated the study of eye movement patterns for sighted people during reading and other visual activities. Today – a newly developed automatic finger tracking system makes it possible to reconstruct blind people’s tactile reading in real time and to automatically analyze finger movements during Braille text reading and tactile picture recognition. In this case study, the very first automatic finger tracking system is presented together with results indicating how Braille readers can increase awareness of their own reading styles. This opens up for future Braille education to become more evidence‐based and, at the same time, for a new research field: contrastive studies of language in its auditory, visual and tactile manifestations.

Computer-based automatic finger- and speech-tracking system

This article presents the first technology ever for online registration and interactive and automatic analysis of finger movements during tactile reading (Braille and tactile pictures). Interactive software has been developed for registration (with two cameras and a microphone), MPEG-2 video compression and storage on disk or DVD as well as an interactive analysis program to aid human analysis. An automatic finger-tracking system has been implemented which also semiautomatically tracks the reading aloud speech on the syllable level. This set of tools opens the way for large scale studies of blind people reading Braille or tactile images. It has been tested in a pilot project involving congenitally blind subjects reading texts and pictures.