Hand Detection Research Articles

Detection of hand motion during cadaveric mastoidectomy dissections: a technical note.

Surgical approaches that access the posterior temporal bone require careful drilling motions to achieve adequate exposure while avoiding injury to critical structures. We assessed a deep learning hand motion detector to potentially refine hand motion and precision during power drill use in a cadaveric mastoidectomy procedure. A deep-learning hand motion detector tracked the movement of a surgeon's hands during three cadaveric mastoidectomy procedures. The model provided horizontal and vertical coordinates of 21 landmarks on both hands, which were used to create vertical and horizontal plane tracking plots. Preliminary surgical performance metrics were calculated from the motion detections. 1,948,837 landmark detections were collected, with an overall 85.9% performance. There was similar detection of the dominant hand (48.2%) compared to the non-dominant hand (51.7%). A loss of tracking occurred due to the increased brightness caused by the microscope light at the center of the field and by movements of the hand outside the field of view of the camera. The mean (SD) time spent (seconds) during instrument changes was 21.5 (12.4) and 4.4 (5.7) during adjustments of the microscope. A deep-learning hand motion detector can measure surgical motion without physical sensors attached to the hands during mastoidectomy simulations on cadavers. While preliminary metrics were developed to assess hand motion during mastoidectomy, further studies are needed to expand and validate these metrics for potential use in guiding and evaluating surgical training.

Thermal Threat Monitoring Using Thermal Image Analysis and Convolutional Neural Networks

Monitoring of the vital signs or environment of disabled people is currently very popular because it increases their safety, improves their quality of life and facilitates remote care. The article proposes a system for automatic protection against burns based on the detection of thermal threats intended for blind or visually impaired people. Deep learning methods and CNNs were used to analyze images recorded by mobile thermal cameras. The proposed algorithm analyses thermal images covering the field of view of a user for the presence of objects with high or very high temperatures. If the user’s hand appears in such an area, the procedure warning about the possibility of burns is activated and the algorithm generates an alarm. To achieve this effect, the thermal images were analyzed using the 15-layered convolutional neural network proposed in the article. The proposed solution provided the efficiency of detecting threat situations of over 99% for a set of more than 21,000 images. Tests were carried out for various network configurations, architecture and both the accuracy and precision of hand detection was 99.5%, whereas sensitivity reached 99.7%. The effectiveness of burn risk detection was 99.7%—a hot object—and the hand appeared simultaneously in the image. The presented method allows for quick, effective and automatic warning against thermal threats. The optimization of the model structure allows for its use with mobile devices such as smartphones and mobile thermal imaging cameras.

High performance PVC/ [AMI]mNTF2 ionic gel sensors for smart wearable applications

Plasticized polyvinyl chloride (PVC) gel as an electroactive polymer material has attracted increasing interest for the construction of flexible devices with the integration of actuation and self-sensing. However, the insufficient sensing ability of traditional PVC gels limits their applications. In this paper, a PVC ionic gel sensor based on the physical blending of PVC gel and the ionic liquid 1-allyl-3-methylimidazolium bisimide ([AMI]mNTF2) is presented. With the addition of ionic liquids, the PVC molecular chains and ions formed stable conductive pathways, resulting in increased conductivity and sensing performance of the PVC gel. The proposed sensor exhibits excellent sensing properties of high resolution (0.12 %) and fast response time (95 ms), along with high sensitivity (GF=27.8), which is more than seven times that of the traditional PVC gel. Furthermore, it was successfully utilized for human motion detection and remote control of a robotic hand, demonstrating the feasibility of the sensor for smart wearable applications in the future.

Hand Gesture Recognition Using Deep Learning

Hand gesture recognition (HGR) has gained significant attention due to its potential for various applications. This paper explores the use of deep learning, specifically Convolutional Neural Networks (CNNs), for HGR using the TensorFlow library. We investigate existing research on CNN-based HGR, focusing on image classification tasks. We then provide a brief overview of CNNs and their suitability for image recognition. Subsequently, we describe the typical workflow of a deep learning-based HGR system, including data preprocessing, hand detection, feature extraction with CNNs, and classification. We highlight the advantages of using TensorFlow to build and train CNN models for HGR. Finally, we conclude by summarizing the key findings from related work and mentioning the specific dataset and number of gestures classified in our research. This work contributes to the growing body of research on CNN-based HGR using TensorFlow and emphasizes its potential for developing accurate and efficient HGR systems.

Hand-object pose estimation based on HOPE-net and attention mechanism

Abstract Hand-object pose estimation is an important part of studying hand pose, focusing on the joint detection of hand and object poses. Although CNN has shown advantages in various hand pose estimation, due to various problems such as the complex structure of the hand pose and self-occlusion, there are limitations in the extraction of image features and the amount of calculation. Existing studies have shown that GCNs have a good performance in dealing with such problems. In this paper, we propose to combine the attention module with two adaptive GCNs, and perform feature enhancement operations on hand features. While ensuring the accuracy of feature extraction, it can effectively improve the network calculation speed and avoid some questions such as self-occlusion. Experimental results show that, through end-to-end training, the algorithm has a great performance on the problem of hand-object pose estimation.

Innovative healthcare solutions: robust hand gesture recognition of daily life routines using 1D CNN.

Hand gestures are an effective communication tool that may convey a wealth of information in a variety of sectors, including medical and education. E-learning has grown significantly in the last several years and is now an essential resource for many businesses. Still, there has not been much research conducted on the use of hand gestures in e-learning. Similar to this, gestures are frequently used by medical professionals to help with diagnosis and treatment. We aim to improve the way instructors, students, and medical professionals receive information by introducing a dynamic method for hand gesture monitoring and recognition. Six modules make up our approach: video-to-frame conversion, preprocessing for quality enhancement, hand skeleton mapping with single shot multibox detector (SSMD) tracking, hand detection using background modeling and convolutional neural network (CNN) bounding box technique, feature extraction using point-based and full-hand coverage techniques, and optimization using a population-based incremental learning algorithm. Next, a 1D CNN classifier is used to identify hand motions. After a lot of trial and error, we were able to obtain a hand tracking accuracy of 83.71% and 85.71% over the Indian Sign Language and WLASL datasets, respectively. Our findings show how well our method works to recognize hand motions. Teachers, students, and medical professionals can all efficiently transmit and comprehend information by utilizing our suggested system. The obtained accuracy rates highlight how our method might improve communication and make information exchange easier in various domains.

Human–Robot Collaborative Manufacturing Cell with Learning-Based Interaction Abilities

This paper presents a collaborative manufacturing cell implemented in a laboratory setting, focusing on developing learning-based interaction abilities to enhance versatility and ease of use. The key components of the system include 3D real-time volumetric monitoring for safety, visual recognition of hand gestures for human-to-robot communication, classification of physical-contact-based interaction primitives during handover operations, and detection of hand–object interactions to anticipate human intentions. Due to the nature and complexity of perception, deep-learning-based techniques were used to enhance robustness and adaptability. The main components are integrated in a system containing multiple functionalities, coordinated through a dedicated state machine. This ensures appropriate actions and reactions based on events, enabling the execution of specific modules to complete a given multi-step task. An ROS-based architecture supports the software infrastructure among sensor interfacing, data processing, and robot and gripper controllers nodes. The result is demonstrated by a functional use case that involves multiple tasks and behaviors, paving the way for the deployment of more advanced collaborative cells in manufacturing contexts.

Multimodal Gesture Recognition with Spatio-Temporal Features Fusion Based on YOLOv5 and MediaPipe

As a natural, intuitive and easy-to-learn mode of interaction, gesture plays an important role in communication. Hand detection, containing multimodal information, includes static and dynamic detection and involves intricate spatial relationship problems such as different hand sizes, complex joints, occlusion and self-occlusion. This study focused on a multimodal hand gesture recognition system based on YOLOv5 and MediaPipe with fused spatio-temporal features. First, the Mediapipe and OpenCV libraries were employed to implement hand keypoint detection. Subsequently, the human–computer interaction (HCI) of volume control was realized by identifying the distance between thumb and index. Finally, model training was conducted based on the YOLOv5 algorithm, and the recognition of different gesture categories was realized. The performance was evaluated and compared through YOLOv5s, YOLOv5m, and YOLOv5l. The gesture recognition system interface visualization was achieved through pyqt5. Experiments show that the average detection accuracy of the model is 99.4% and the recognition speed is around 0.2[Formula: see text]s.

A Dataset for Kinship Estimation from Image of Hand Using Machine Learning

Kinship (Familial relationships) detection is crucial in many fields and has applications in biometric security, adoption, forensic investigations, and more. It is also essential during wars and natural disasters like earthquakes since it may aid in reunion, missing person searches, establishing emergency contacts, and providing psychological support. The most common method of determining kinship is DNA analysis which is highly accurate. Another approach, which is noninvasive, uses facial photos with computer vision and machine learning algorithms for kinship estimation. Each part of the Human -body has its own embedded information that can be extracted and adopted for identification, verification, or classification of that person. Kinship recognition is based on finding traits that are shared by every family. We investigate the use of hand geometry for kinship detection, which is a new approach. Because of the available hand image Datasets do not contain kinship ground truth; therefore, we created our own dataset. This paper describes the tools, methodology, and details of the collected MKH, which stands for the Mosul Kinship Hand, images dataset. The images of MKH dataset were collected using a mobile phone camera with a suitable setup and consisted of 648 images for 81 individuals from 14 families (8 hand situations per person). This paper also presents the use of this dataset in kinship prediction using machine learning. Google MdiaPipe was used for hand detection, segmentation, and geometrical key points finding. Handcraft feature extraction was used to extract 43 distinctive geometrical features from each image. A neural network classifier was designed and trained to predict kinship, yielding about 93% prediction accuracy. The results of this novel approach demonstrated that the hand possesses biometric characteristics that may be used to establish kinship, and that the suggested method is a promising way as a kinship indicator.

Enhancing Road Safety: A Driver Fatigue Detection and Behaviour Monitoring System using Advanced Computer Vision Techniques

Driver drowsiness is a major hazard to road safety, necessitating the development of reliable detection technologies. This study describes a revolutionary driver fatigue detection system that uses cutting-edge computer vision technologies. This system uses the MediaPipe framework for accurate face and hand detection and the Eye Aspect Ratio (EAR) for drowsiness detection. Furthermore, it uses the OpenCV solvePnP function for estimating rotation vector, and converting it to a rotation angle of the head, to check driver attention. By continuously monitoring these indicators, when the system successfully detects any instances of driver tiredness or inattention, it records his or her behavior and delivers notifications to help prevent accidents. This study helps to improve road safety by utilizing cutting-edge computer vision techniques to prevent driver fatigue and boost attentive driving practices.

Object and person tracking systems to enable extremity dosimetry in nuclear medicine using computational methods

Nuclear medicine (NM) professionals are potentially exposed to high doses of ionising radiation, particularly in the skin of the hands. Ring dosimeters are used by the workers to ensure extremity doses are kept below the legal limits. However, ring dosimeters are often susceptible to large uncertainties, so it is difficult to ensure a correct measurement using the traditional occupational monitoring methods. An alternative solution is to calculate the absorbed dose by using Monte Carlo simulations. This method could reduce the uncertainty in dose calculation if the exact positions of the worker and the radiation source are represented in these simulations. In this study we present a set of computer vision and artificial intelligence algorithms that allow us to track the exact position of unshielded syringes and the hands of NM workers. We showcase a possible hardware configuration to acquire the necessary input data for the algorithms. And finally, we assess the tracking confidence of our software. The tracking accuracy achieved for the syringe detection was 57% and for the hand detection 98%.

SignEase - Sign Language Interpreter Model An Indian Sign Language Interpretation Model using Machine Learning and Computer Vision Technology

Abstract: This study introduces a real-time system designed to recognize hand poses and gestures from the Indian Sign Language (ISL) using grid-based (control points) features. The primary aim is to bridge communication barriers between the hearing and speech impaired individuals and the broader society.Existing solutions often struggle with either accuracy or realtime performance, whereas our system excels in both aspects. It can accurately identify hand gestures in Indian Sign Language. In addition to recognition capabilities, our system offers a ’Learning Portal’ for users to efficiently learn and practice ISL, ASL, etc., enhancing its accessibility and effectiveness. Notably, the system operates solely on smartphone camera input, eliminating the need for any external hardware like gloves or specialized sensors, thus ensuring user-friendliness. Key techniques employed include hand detection via MediaPipe, cvzone, etc modules, and grid-based feature extraction, which transforms hand poses into concise feature vectors. These features are then compared with a TensorFlow- provideddatabase for classification for accurate translation

Enabling near-real-time safety glove detection through edge computing and transfer learning: comparative analysis of edge and cloud computing-based methods

PurposeComputer vision and deep learning (DL) methods have been investigated for personal protective equipment (PPE) monitoring and detection for construction workers’ safety. However, it is still challenging to implement automated safety monitoring methods in near real time or in a time-efficient manner in real construction practices. Therefore, this study developed a novel solution to enhance the time efficiency to achieve near-real-time safety glove detection and meanwhile preserve data privacy.Design/methodology/approachThe developed method comprises two primary components: (1) transfer learning methods to detect safety gloves and (2) edge computing to improve time efficiency and data privacy. To compare the developed edge computing-based method with the currently widely used cloud computing-based methods, a comprehensive comparative analysis was conducted from both the implementation and theory perspectives, providing insights into the developed approach’s performance.FindingsThree DL models achieved mean average precision (mAP) scores ranging from 74.92% to 84.31% for safety glove detection. The other two methods by combining object detection and classification achieved mAP as 89.91% for hand detection and 100% for glove classification. From both implementation and theory perspectives, the edge computing-based method detected gloves faster than the cloud computing-based method. The edge computing-based method achieved a detection latency of 36%–68% shorter than the cloud computing-based method in the implementation perspective. The findings highlight edge computing’s potential for near-real-time detection with improved data privacy.Originality/valueThis study implemented and evaluated DL-based safety monitoring methods on different computing infrastructures to investigate their time efficiency. This study contributes to existing knowledge by demonstrating how edge computing can be used with DL models (without sacrificing their performance) to improve PPE-glove monitoring in a time-efficient manner as well as maintain data privacy.

Sign Language Recognition Using Convolutional Neural Network

This project introduces a real-time sign language detection system powered by Convolutional Neural Networks (CNNs), designed to aid individuals with hearing impairments in communication. Leveraging OpenCV for video capture and hand detection, along with custom modules for hand tracking and image classification, the system seamlessly integrates deep learning methodologies. A pre-trained CNN model, trained on a comprehensive dataset of sign language gestures, forms the core of the system, ensuring accurate classification in real-time. By capturing video frames from a webcam, detecting hands, and processing them through the CNN model, the system provides immediate feedback by overlaying predicted gestures onto the video stream. Through its effective implementation, this project underscores the potential of CNNs in facilitating accessibility and inclusivity, while also paving the way for future enhancements and applications in assistive technology and human-computer interaction. Top of Form Keywords— Real-time, Sign Language Detection, Convolutional Neural Networks (CNNs), Communication, Hearing Impairments, Deep Learning.

228 Expert Hand Motion During Microvascular Anastomosis Simulation: Big Data Analysis Using Machine Learning Hand Detection

INTRODUCTION: Microanastomosis is a complex surgical technique that demands delicate and coordinated hand movement. Advances in machine learning technology have enabled new methods to study human motion. Convolutional neural networks are algorithms specialized in visual tasks that enable discrete object detection and tracking including the hands of a surgeon. METHODS: A machine learning hand detector was developed using a pre-trained convolutional neural network capable of tracking 21 hand landmarks without sensors attached. En-to-side microanastomosis was simulated using 2 mm polyvinyl-alcohol vessels and 10/0 micro-suture and recorded with a video camera capturing hand movement in the surgical field. Participants (n = 4) were expert cerebrovascular surgeons with significant operative experience in cerebral bypass surgery. Calculation of motion of 14 hand joint angles was performed using a vectorization method from landmark detections. Time series analysis was performed to calculate the absolute angular motion for each joint of the hand. RESULTS: Four minutes of simulation for each surgeon yielded a total of 784.294 angular detections (14 hand joints total). For the dominant hand, the mean (SD) absolute angular motion was expert 1: 178.2 rad (98.3); expert 2: 261.8 rad (164.3); expert 3: 449.9 rad (307); expert 4: 380.1 rad (198.1).For the non-dominant hand, the mean (SD) absolute angular motion was expert 1: 132.9 rad (64.6), expert 2: 149.2 rad (72.3); expert 3: 389.2 rad (243); expert 4: 162.6 rad (87.2). CONCLUSIONS: A machine learning hand detector system can calculate hand joint motion during microvascular anastomosis. The absolute angular motion is a quantitative performance metric providing information about hand stability during microanastomosis simulation. Such studies using powerful artificial intelligence detection algorithms may have educational potential in the assessment of complex and delicate surgical skills and supervision of surgical training.

Virtual Air Canvas: A Review

This creative method of utilizing open-source frameworks to create gesture-controlled whiteboard devices is impressive. The device tracks and interprets finger movements collected by a camera to allow users to engage with a virtual whiteboard. The implementation includes effective data processing, hand landmark estimate, and real-time hand detection by utilizing computer vision algorithms. By providing a modern form of communication and going beyond regular typing and writing habits, the system seeks to expand beyond traditional writing methods and create new opportunities for connection. The system contributes to the changing field of human-computer interaction by offering an alternate mode of communication, which improves people's quality of life overall. It also demonstrates the versatility of gesture-controlled systems in terms of enhancing communication accessibility. Key Words: Data Processing, Handmark Estimation, Computer Vision algorithms, MediaPipe, OpenCV.

Sign Language Recognition by Image Processing

A prototype sign language interpreter that can converse in American Sign Language (ASL) is presented in this study. The visual stimulation of sign language aids in the development of speech and language. It promotes social contact, fosters the development of cognitive structures, and lessens harmful social behaviors. Early exposure to sign language supports language development in academic, social, and emotional domains. Building a user-hand gesture-only human-computer interface (HCI) is the foundation of the dependable paradigm. Using Python and OpenCV, hand gesture recognition is built using the theories of hand segmentation and the hand detection system. The training photos are labeled with different names based on what is in the database. A binary feature vector is created from each image. Convolutional neural networks (CNNs) and cameras are used to capture images from hand sign videos in real time, which are then used to carry out the feature extraction process. The algorithm generates output based on the projected maximum similarity between these features and the features of database photographs. With the aid of an application, this model provides adequate accuracy even in the absence of a constant or monochromatic background. The person's hand movements are captured on video with the camera; the video is subsequently processed and translated into text. The user hears the speech that has been transformed from the output. The suggested system recognizes letters, numbers, and certain common words in sign language. Raw photos and videos are transformed into corresponding text that can be read and comprehended by using image processing algorithms and neural networks to map the gesture to the right text in the training data. When paired with a sizable source database, this protocol will surely be highly beneficial for closing the communication gap between people who can talk and hear and those who cannot.

Keyboard and Mouse Free Music Controller

Abstract: Human-machine interface (HMI) is a crucial area of research as gestures have the potential to efficiently control and interact with computers. Many applications for hand detection have been created as a result of the pervasive use of built-in cameras in computers, smartphones, and tablets. For the majority of users, however, many of these are not useful. A straightforward concept for a keyboard- and mouse-free music controller is presented in this research. Using MATLAB code that integrates skin detection, area labelling, erosion, dilation, and motion differentiation, a music player controller is developed using the real-time frame tracking feature of a camera. Three hand detection algorithms are created and assessed for maximum performance and accuracy. Real-time hand detection for operating the music player is provided by the algorithm, which is created with efficiency and speed in mind.

A Hand Gesture-Operated System for Rehabilitation Using an End-to-End Detection Framework

We propose a hand gesture-operated system as an AI application to relieve discomfort and restore function in hand and arm movements caused by injuries and nerve and muscle complications. The system trains patients with hand exercises, such as performing hand gestures accurately, traversing within specified bounds, and operating a hand gesture calculator. However, the system requires accurate hand gesture detection, which is impeded by background clutter and variations in illumination and in the hand's size and angle. To address this, we developed a robust hand detection module that uses a single-stage transformer deep network. The transformer network encodes global information and uses bipartite matching to reduce the frequency of spurious detections. It drives a regression head and a classification head to localize the hand gesture in a bounding box and assign it a class label. Hand keypoints are also detected to support drawing, path traversal, and calculator use. The approach is evaluated on two benchmark datasets: OUHANDS and NUS. The method yields 89.6% accuracy for OUHANDS and 100% for NUS. These results indicate that precise hand detection can support a robust system for rehabilitation through hand exercises. Our experiments confirm that the users' hand function progressively improved.

Designing a 3D gestural interface to support user interaction with time-oriented data as immersive 3D radar charts

The design of intuitive three-dimensional user interfaces is vital for interaction in virtual reality, allowing to effectively close the loop between a human user and the virtual environment. The utilization of 3D gestural input allows for useful hand interaction with virtual content by directly grasping visible objects, or through invisible gestural commands that are associated with corresponding features in the immersive 3D space. The design of such interfaces remains complex and challenging. In this article, we present a design approach for a three-dimensional user interface using 3D gestural input with the aim to facilitate user interaction within the context of Immersive Analytics. Based on a scenario of exploring time-oriented data in immersive virtual reality using 3D Radar Charts, we implemented a rich set of features that is closely aligned with relevant 3D interaction techniques, data analysis tasks, and aspects of hand posture comfort. We conducted an empirical evaluation (n=12)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$(n=12)$$\\end{document}, featuring a series of representative tasks to evaluate the developed user interface design prototype. The results, based on questionnaires, observations, and interviews, indicate good usability and an engaging user experience. We are able to reflect on the implemented hand-based grasping and gestural command techniques, identifying aspects for improvement in regard to hand detection and precision as well as emphasizing a prototype’s ability to infer user intent for better prevention of unintentional gestures.