Data Glove Research Articles

Mass-Produced Flexible Strain Sensors Based on Dip-Coating and Water Bath for Human–Computer Interaction

Flexible strain sensors play a key role in the field of human–machine interaction because of their ability to sense deformation. To meet the needs of mass production of wearable electronic devices, it is very necessary to develop a strain sensor with excellent sensing performance and low-cost mass fabrication. In this article, a fabrication process of a flexible resistive strain sensor, which is perfectly matched by the combined method with dip-coating and water bath, is proposed. A layer of conductive polymers containing carbon black/multiwall carbon nanotubes/thermoplastic polyurethane (CB/MWCNTs/TPU) was quickly and evenly wrapped on the surface of the core-spun elastic yarn. Combining the synergistic conductive effect of the filler and the high elasticity of the yarn matrix, the designed flexible strain sensor has high sensitivity (gauge factor (GF) to 68), low detection limit, wide strain range, fast response time, fast recovery time, and high stability. The designed strain sensor exhibits outstanding sensing performance and wearability, which can be used for real-time monitoring of human joint bending and breathing state. Furthermore, a wireless operating system based on wearable data gloves has been developed to realize the remote synchronous movement of the human hand/the bionic manipulator. The results indicate that the proposed strain sensor has great potential in human–machine interaction and its fabrication technology provides guidance for the sensor in mass production.

State-of-the-Art in Data Gloves: A Review of Hardware, Algorithms, and Applications

Data glove is one of the devices that can express human intention via detecting hand motion with high-precision. In recent years, flourished applications have been driven by data gloves from human-machine interaction (HMI) to biomedical sciences. To understand the latest research status and grasp the development direction, this study provides a systematic overview of the state-of-the-art for data gloves. Specifically, we take the research topics of hardware design, algorithms for gesture recognition, and the application areas as the entry point. Detailed elaboration of key algorithms techniques, from data processing to classification, for hand gesture recognition is made and quantitative analysis of the recent research achievements is summarized. We found that hand motion detection with data gloves is mostly employed for gesture recognition, and the majority of studies are focused on the performance validation of the gesture recognition, in particular for static gestures. A systematic illustration of the current research challenges as well as the future directions of related data gloves are finally discussed.

LiDAR-Based Hand Contralateral Controlled Functional Electrical Stimulation System.

Contralateral controlled functional electrical stimulation (CCFES) can induce simultaneous movements in patients' bilateral hands. It has been clinically proven to be effective in improving hand motor control and dexterity. sEMG and bending sensor-based data gloves for detecting patients' motor intent have been developed with limitations. sEMG sensor signals are unstable and susceptible to noise. Data gloves composed of bending sensors require complicated calibration and tend to have data drift. In this paper, a LiDAR-based system for hand CCFES is proposed. The method utilized LiDAR to detect the patient's motion intention without contact in CCFES systems. It has been clinically proven that LiDARs can effectively distinguish the different motion amplitudes of hand gestures as quantitative evaluation sensors of functional electrical stimulation (FES). Training data for classifiers were collected from 9 healthy individuals and 15 stroke patients performing 4 gestures, including hand opening, fist clenching, wrist extension, and wrist flexion. The support vector machine (SVM), linear discriminant analysis (LDA), and k-nearest neighbor (kNN) were verified for their classification performance in offline hand gesture recognition tests. Experiments were also conducted on 6 stroke volunteers to evaluate gestures triggered by FES. The SVM classifier showed excellent classification performance for four hand gestures, with an average F1-score of 0.97 ± 0.05 in offline tests. As for online gesture recognition, an average F1-score of 0.92 ± 0.09 was obtained. In the evaluation experiments, between data from 50% and 100% movement amplitude, paired t-tests showed significant differences. The experimental results indicated that the proposed system showed promise for hand rehabilitation.

Data Glove with Self-Compensation Mechanism Based on High-Sensitive Elastic Fiber-Optic Sensor

With the development of virtual reality (VR) interaction technology, data glove has become one of the most popular devices for human-computer interaction. It's valuable to design high-sensitive and flexible sensor for data glove. Therefore, a low-cost data glove based on self-compensating elastic optical fiber sensor with self-calibration function is proposed. The tunable and stretchable elastic fiber was fabricated by a simple, economical and controllable method. The fiber has good flexibility and high stability under stretching, bending and indentation deformation. The optical fibers are installed in the sensor in a U shape with a bending radius of 5 mm. Compared with the straight fiber, the response sensitivity of the U-shaped fiber to deformation is increased by about 7 times at most. The reference optical fiber is connected to the sensor, which effectively improves the stability and accuracy of the sensor system. In addition, the sensors are easy to install so that the data gloves can be customized for different hand shapes. In the gesture capture test, it can respond quickly and guide the manipulator to track the gesture. This responsive and stable data glove has broad development potential in motion monitoring, telemedicine and human-computer interaction.

End-Side Gesture Recognition Method for UAV Control

Traditional heavy and complex control equipment such as remote controllers (RCs) and ground stations (GSs) are not able to meet the fast and flexible control requirements of unmanned aerial vehicles (UAVs) in complex environments. This article thus proposes a gesture recognition method for the control of UAVs. To achieve the goal of gestural manipulation of UAVs, this method can accurately recognize a variety of gestures and convert the gestures into corresponding UAV control commands. First, the wireless data glove incorporates flex sensors, and inertial sensors are used to gather gesture datasets. The trained neural network model is then deployed on the data glove based on an STM32 microcontroller for real-time gesture recognition, in which the backpropagation (BP) network is used for static gesture recognition and the bidirectional gated recurrent unit (Bi-GRU) network is used for dynamic gesture recognition. Finally, the gesture is converted into the control command and sent to the aircraft terminal to control the UAV. Using the UAV simulation test on the simulation platform, the recognition accuracy of the basic UAV control commands corresponding to ten static gestures is found to be 100%, and the mean recognition accuracy of the UAV mode-switching commands corresponding to five dynamic gestures is 98.4% indicating that the gesture recognition effect of the system is perfect. Mission testing performed in the scene constructed in the simulation environment demonstrates that the UAV can respond rapidly to gestures, and the method proposed in this article can achieve real-time and stable control of the UAV on the end side.

An Improved Gesture Recognition Model Based on Mini-Xception

With the rapid development of artificial intelligence technology, gestures have become the mainstream in the field of human-computer interaction because of their simplicity, easy understanding and non-contact. Compared with the early data gloves, the vision-based non-contact gesture recognition interaction method has obvious advantages. However, the variability of the gesture itself, the complexity of the background and the influence of different lighting conditions have impacted the accuracy of gesture recognition. With the rapid development of deep learning technology, gesture recognition has achieved amazing results in accuracy. To improve gesture recognition accuracy in complex background, this paper optimizes the mini-xception network model of lightweight convolutional neural network algorithm, introduce the transfer learner and integrate YOLOV4-tiny into the mini-xception model, generate a new network model YT_ mini-Xception. After experimental verification, by YT_ Mini-xception network model, the average accuracy of 0-9 gesture recognition on complex background data set is 96.64%, the average recognition time was 39.8 milliseconds, and the expected goal was achieved.

Hand Gesture based Speech Recognition system for Hard of Hearing People

Abstract: For people who are intellectually challenged, this document provides a sign to speech translator. In contemporary society, the dumb have very little ability to converse with the normal person. Therefore, they are unable to interact with common people until common people, like ourselves, learn how to communicate through sign language. Dumb sign language is not something that everyone can learn because it is so challenging to perfect. Because of this, nobody can go and talk to these physically challenged persons. So, here is a system that would make it possible for those who are illiterate to speak with everyone. With today's technology, even the deaf may communicate by making hand gestures. Each hand gesture is set to play back a different audio message. Hand motions were captured by a flex sensor and an accelerometer, and the microcontroller then accepted these inputs. The flex sensor tracks hand movements and detects deflections, converting them into the right signals that trigger the system to produce recorded sound signals for transmission via the speakers. There are many different Arduino Boards available right now, and they can be used for a variety of things. The Arduino Mega 2560 is the ideal solution since it is the most cost-effective, allows faster computing, and can integrate and interface various sensors while consuming less power. The device under discussion is basically made up of a glove and a microcontroller-based system. Four strategically placed flex sensors are included in the data glove.

A virtual surgical prototype system based on gesture recognition for virtual surgical training in maxillofacial surgery

BackgroundVirtual reality (VR) technology is an ideal alternative of operation training and surgical teaching. However, virtual surgery is usually carried out using the mouse or data gloves, which affects the authenticity of virtual operation. A virtual surgery system with gesture recognition and real-time image feedback was explored to realize more authentic immersion.MethodGesture recognition technology proposed with an efficient and real-time algorithm and high fidelity was explored. The recognition of hand contour, palm and fingertip was firstly realized by hand data extraction. Then, an Support Vector Machine classifier was utilized to classify and recognize common gestures after extraction of feature recognition. The algorithm of collision detection adopted Axis Aligned Bounding Box binary tree to build hand and scalpel collision models. What’s more, nominal radius theorem (NRT) and separating axis theorem (SAT) were applied for speeding up collision detection. Based on the maxillofacial virtual surgical system we proposed before, the feasibility of integration of the above technologies in this prototype system was evaluated.ResultsTen kinds of signal static gestures were designed to test gesture recognition algorithms. The accuracy of gestures recognition is more than 80%, some of which were over 90%. The generation speed of collision detection model met the software requirements with the method of NRT and SAT. The response time of gesture] recognition was less than 40 ms, namely the speed of hand gesture recognition system was greater than 25 Hz. On the condition of integration of hand gesture recognition, typical virtual surgical procedures including grabbing a scalpel, puncture site selection, virtual puncture operation and incision were carried out with realization of real-time image feedback.ConclusionBased on the previous maxillofacial virtual surgical system that consisted of VR, triangular mesh collision detection and maxillofacial biomechanical model construction, the integration of hand gesture recognition was a feasible method to improve the interactivity and immersion of virtual surgical operation training.

STPA systemic approach for OHS and operational risk analysis of data glove use in 4.0 assembly

The complexity of manufacturing is rising considerably and it enhances through the introduction of new technologies to the system. Advancements in new technologies (in this study data gloves) and their application in the 4.0 assembly context will bring about new and emerging occupational health and safety (OHS) and operational risks. This study aims to apply STAMP (Systems-Theoretic Accident Model and Processes)-based approach to analyze these risks when a data glove is introduced to an assembly system. The application of STPA (Systems-Theoretic Process Analysis) on realistic case studies provides a systemic view of systems’ control structure from OHS and operational concerns. Findings indicate that STPA is a promising method for OHS and operational risk analysis when a wearable is introduced to a complex assembly system, especially in designing step. However, technology readiness level limits access to numerical and detailed data in providing insight into intensity of the different control structure’s effect and system components and their interactions on the studied system.

Study on A Mechanical Data Glove

Visual recognition, point tracking recognition and micro-sensor technology can be used to monitor the hand posture, but they all have some shortcomings, such as expensive system, complex algorithm and difficult processing. To solve these problems, this paper designs a mechanical data glove based on variable resistor. The finger bending information is collected through the relationship between the output of potentiometers and hand posture, and the finger bending information is converted into digital signal, which can be displayed again after processing. The data glove not only has simple structure and low cost, but also can effectively monitor the hand posture.

Low‐Cost Data Glove Based on Deep‐Learning‐Enhanced Flexible Multiwalled Carbon Nanotube Sensors for Real‐Time Gesture Recognition

With advancements in artificial intelligence, wearable motion recognition systems based on flexible nanomaterial sensors exhibit excellent potential for harmonious human–machine interaction. However, the sensing stability and demand of large‐scale arrays limit the application of flexible nanomaterial sensors. Herein, a data glove system based on simple multiwalled carbon nanotube (MWCNT) sensors and a lightweight deep‐learning algorithm to achieve accurate gesture recognition is proposed. A regional‐crack mechanism is introduced through the microspine structure to enhance the strain sensitivity. Moreover, an efficient signal processing strategy based on an adaptive wavelet threshold function to improve the robustness and anti‐interference of signals obtained from MWCNT sensors, which exhibit strong generalization and can be used in other nanomaterial strain sensor. Based on the depth‐wise separable convolution, a novel hybrid convolutional neural network (CNN) long short‐term memory (LSTM) model for gesture recognition is constructed. The proposed model achieves an average accuracy of 97.5% and recognition accuracy of 30 gestures with an average recognition time of 2.173 ms based on only five sensors. The fabricated data glove is a promising platform for low‐cost and wearable human–machine interaction that can be directly interfaced in applications such as robotic hands, smart cars, and first‐person shooting games.

Estimating Finger Joint Motions Based on the Relative Sliding of Layered Belts

This article introduces a new method for measuring finger joints’ motion based on the relative sliding motions of belts placed on the finger. Conventional mechanical methods used in wearable devices for measuring hand motions, such as a method using resistive sensors, require calibration for the difference in the fingers’ shapes of individuals. In contrast, the proposed method measures the metacarpophalangeal (MP), proximal interphalangeal (PIP), and distal interphalangeal (DIP) joints flex motion of a finger except for the thumb without the above individual calibration using the difference in the sliding amount of belts on the finger surface. The simple equations show that the relative sliding amount of belts is proportional to joint angles with a factor of the belt thickness. Therefore, the method only need to know the belts’ thickness beforehand to calculate the joint angles from the relative belt positions but not the fingers’ shapes. Experiments with a prototype showed that the proposed method estimated the joint motion of the finger model driven by servomotors with an accuracy of less than 1° in three kinds of motions. The device was in an early stage of development as a data glove, but the accuracies were comparable to other wearable devices. The layered belts mechanism proposed here can provide an option to design a device to realize simplified hand motion measurement by eliminating cumbersome calibration processing for each execution. Furthermore, the simple measurement principle also has an advantage in robustness to noise, such as the drift seen in the devices using inertial measurement units (IMUs).

Design of a Multi-Sensor System for Exploring the Relation between Finger Spasticity and Voluntary Movement in Patients with Stroke.

A novel wearable multi-sensor data glove system is developed to explore the relation between finger spasticity and voluntary movement in patients with stroke. Many stroke patients suffer from finger spasticity, which is detrimental to their manual dexterity. Diagnosing and assessing the degrees of spasticity require neurological testing performed by trained professionals to estimate finger spasticity scores via the modified Ashworth scale (MAS). The proposed system offers an objective, quantitative solution to assess the finger spasticity of patients with stroke and complements the manual neurological test. In this work, the hardware and software components of this system are described. By requiring patients to perform five designated tasks, biomechanical measurements including linear and angular speed, acceleration, and pressure at every finger joint and upper limb are recorded, making up more than 1000 features for each task. We conducted a preliminary clinical test with 14 subjects using this system. Statistical analysis is performed on the acquired measurements to identify a small subset of features that are most likely to discriminate a healthy patient from patients suffering from finger spasticity. This encouraging result validates the feasibility of this proposed system to quantitatively and objectively assess finger spasticity.

Low-Cost Self-Calibration Data Glove Based on Space-Division Multiplexed Flexible Optical Fiber Sensor.

Wearable devices such as data gloves have experienced tremendous growth over the past two decades. It is vital to develop flexible sensors with fast response, high sensitivity and high stability for intelligent data gloves. Therefore, a tractable low-cost flexible data glove with self-calibration function based on a space-division multiplexed flexible optical fiber sensor is proposed. A simple, stable and economical method was used to fabricate flexible silicone rubber fiber for a stretchable double-layered coaxial cylinder. The test results show that the fiber is not sensitive to the temperature range of (20~50 °C) and exhibits excellent flexibility and high stability under tensile, bending and torsional deformation. In addition, the signal detection part of the data glove enables compact and efficient real-time information acquisition and processing. Combined with a self-calibration function that can improve the accuracy of data acquisition, the data glove can be self-adaptive according to different hand sizes and bending habits. In a gesture capture test, it can accurately recognize and capture each gesture, and guide the manipulator to make the same action. The low-cost, fast-responding and structurally robust data glove has potential applications in areas such as sign language recognition, telemedicine and human–robot interaction.

Interactive Application of Data Glove Based on Emotion Recognition and Judgment System.

In this paper, the interactive application of data gloves based on emotion recognition and judgment system is investigated. A system of emotion recognition and judgment is established based on the set of optimal features of physiological signals, and then a data glove with multi-channel data transmission based on the recognition of hand posture and emotion is constructed. Finally, the system of virtual hand control and a manipulator driven by emotion is built. Five subjects were selected for the test of the above systems. The test results show that the virtual hand and manipulator can be simultaneously controlled by the data glove. In the case that the subjects do not make any hand gesture change, the system can directly control the gesture of the virtual hand by reading the physiological signal of the subject, at which point the gesture control and emotion control can be carried out at the same time. In the test of the manipulator driven by emotion, only the results driven by two emotional trends achieve the desired purpose.

Discriminating Free Hand Movements Using Support Vector Machine and Recurrent Neural Network Algorithms.

Decoding natural hand movements is of interest for human–computer interaction and may constitute a helpful tool in the diagnosis of motor diseases and rehabilitation monitoring. However, the accurate measurement of complex hand movements and the decoding of dynamic movement data remains challenging. Here, we introduce two algorithms, one based on support vector machine (SVM) classification combined with dynamic time warping, and the other based on a long short-term memory (LSTM) neural network, which were designed to discriminate small differences in defined sequences of hand movements. We recorded hand movement data from 17 younger and 17 older adults using an exoskeletal data glove while they were performing six different movement tasks. Accuracy rates in decoding the different movement types were similarly high for SVM and LSTM in across-subject classification, but, for within-subject classification, SVM outperformed LSTM. The SVM-based approach, therefore, appears particularly promising for the development of movement decoding tools, in particular if the goal is to generalize across age groups, for example for detecting specific motor disorders or tracking their progress over time.

Sign annotation generation to alphabets via integrating visual data with somatosensory data from flexible strain sensor-based data glove

Sign language forms a significant way of communication for deaf-mute people. There are different types of sign languages of different countries in different regions. The sign language recognition system translates the gestures into the alphabet, which solves a barrier in comprehension of conversation. Visual techniques and somatosensory strain sensors are both useful modalities to detect sign gesture. In order to improve the recognition accuracy and wear resistance, this paper proposes a real-time alphabets conversion system, which combines the visual data with the data glove based on flexible strain sensor and the somatosensory data of the camera, supplemented by deep learning. In this learning structure, convolutional neural network is used to process the visual data, and then the sparse neural network for sensor data fusion and recognition is realized in the feature layer. A module is proposed to train the linking of hand motions primitives to alphabets (A–Z). The proposed framework is tested on a dataset of sign and descriptive alphabets. The proposed system would recognize the 26 letters with a recognition rate of up to 99.50% and a recognition time of less than 1 s, which helps to understand observations of sign as words and texts.

Reducing the Number of Sensors in the Data Glove for Recognition of Static Hand Gestures

Data glove devices, apart from being widely used in industry and entertainment, can also serve as a means for communication with the environment. This is possible thanks to the advancement in electronic technology and machine learning algorithms. In this paper, the results of the study using a designed data glove equipped with 10 piezoelectric sensors are reported, and the designed glove is validated on a recognition task of hand gestures based on 16 static signs of the Polish Sign Language (PSL) alphabet. The main result of the study is that recognition of 16 PSL static gestures is possible with a reduced number of piezoelectric sensors. This result has been achieved by applying the decision tree classifier that can rank the importance of the sensors for the recognition performance. Other machine learning algorithms were also tested, and it was showed that for the Support Vector Machines, k-NN and Bagged Trees classifiers, a recognition rate of the signs exceeding 90% can be achieved just for three preselected sensors. Such a result is important for a reduction in design complexity and costs of such a data glove with sustained reliability of the device.

Virtual interaction and manipulation control of a hexacopter through hand gesture recognition from a data glove

AbstractThe purpose of this study is to realize virtual interaction and manipulation control of a hexacopter based on hand gesture recognition from a designed data glove, to provide an intuitive and visual real-time simulation system for flight control algorithm verification and external control equipment testing. First, the hand gesture recognition from a designed data glove is studied, which can recognize different actions, such as mobile ready, grab, loosen, landing, take-off, and hover. Then, the design of virtual simulation system for hexacopter capture is completed, with the model design of hexacopter and manipulator, and the simulation software design with $CoppeliaSim$ . Finally, virtual simulation experiment of hexacopter grasping and virtual flight control experiment based on data glove are tested, respectively, and quantitatively described. The overall recognition rate is 84.3%, indicating that the data glove produced has the ability to recognize gestures, but its recognition performance is not superior. In gesture recognition, the recognition rate of static gestures is relatively higher than that of dynamic gestures. Among the static gestures, the hover gesture has the highest recognition rate. The average correct rate of static gestures can reach 94%. The lowest recognition rate of dynamic gestures is upward movement, and the average recognition rate of dynamic gestures is 76.1%. The research can be used to remotely operate hexacopter using a data glove in the future and improve the control performance through virtual interaction and manipulation simulation before actual application.

MFA: A Smart Glove with Multimodal Intent Sensing Capability.

At present, virtual-reality fusion smart experiments mostly employ visual perception devices to collect user behavior data, but this method faces the obstacles of distance, angle, occlusion, light, and a variety of other factors of indoor interactive input devices. Moreover, the essence of the traditional multimodal fusion algorithm (TMFA) is to analyze the user's experimental intent serially using single-mode information, which cannot fully utilize the intent information of each mode. Therefore, this paper designs a multimodal fusion algorithm (hereinafter referred to as MFA, Algorithm 4) which focuses on the parallel fusion of user's experimental intent. The essence of the MFA is the fusion of multimodal intent probability. At the same time, this paper designs a smart glove based on the virtual-reality fusion experiments, which can integrate multichannel data such as voice, visual, and sensor. This smart glove can not only capture user's experimental intent but also navigate, guide, or warn user's operation behaviors, and it has stronger perception capabilities compared to any other data glove or smart experimental device. The experimental results demonstrate that the smart glove presented in this paper can be widely employed in the chemical experiment teaching based on virtual-reality fusion.