Recent advancements in multimedia technology have revolutionized digital device interactions, with sign language recognition emerging as a crucial tool for improving accessibility. This article introduces an innovative approach to integrating language-based sign language recognition into multimedia applications, enabling automatic recognition of gestures across various nationalities in dynamic sign language environments (video). Our research explores the potential of this technology to enhance accessibility and usability, particularly for individuals with hearing impairments, by facilitating intuitive and real-time control over video conferencing applications. We identify existing challenges in multimedia applications and propose a novel framework incorporating sign language recognition algorithms. Our approach involves developing a prototype multimedia application designed for fast communication in crowded environments and backgrounds. Feature extraction is performed using the Mediapipe Holistic framework, which captures hand-shoulder distances, finger angles, and finger usage. Gesture classification is achieved using the K-Nearest Neighbors (KNN) algorithm, effectively recognizing international sign language gestures. Additionally, for language and word prediction, we employ Convolutional Recurrent Neural Networks (CRNNs) enhanced by Long Short-Term Memory (LSTM) to process diverse linguistic contexts. Experimental results confirm the robustness of our approach, achieving 89% accuracy in multiclass gesture classification and exceeding 93% accuracy in word prediction across a large, diverse dataset collected from multiple sources and languages. To enhance practical applicability, we integrated our techniques into a custom video conferencing application built with Django. This application seamlessly incorporates our feature extraction and prediction models, offering an innovative, two-way communication platform with improved accessibility features.

Lightweight hand motion mechanism was developed to achieve high speed pitching. It is indicated that humans have increased fingers stiffness when putting a spin on a ball. By focusing on this feature, fingers have been made motor-less and lightweight. Specifically, the mechanism that fixing PIP angles of index and middle fingers when grasping a ball and switching to elastic joints by release fixation when putting a spin on the ball was developed. Stiffness was considered from a finger model when pitching a fast ball. Pitching experiments confirmed that the robot was able to grasp a ball and put a spin on the ball.

This study addresses key challenges in upper limb rehabilitation for stroke patients, including the dependence on third-party assistance, the high cost of smart devices, and the lack of multisensory stimulation in traditional methods. This paper proposes a 3D maze rehabilitation game system based on gesture recognition. The system is built on a dual architecture consisting of a Python-based gesture recognition endpoint and a Unity-based game interaction endpoint. To reduce costs, the hardware relies on a standard computer camera, eliminating the need for depth sensors. On the software side, the MediaPipe framework is employed to track 21 3D hand keypoints in real-time. Three feature types, relative coordinates, finger angles, and relative distances, are extracted and normalized to mitigate distance and scale interference. Using publicly available datasets and an Support Vector Machine algorithm, this study developed a model to classify 10 distinct gestures. The system ensures cross-platform communication via UDP, and Unity is used to create dual scenes, fully mapping gestures to game controls, enabling patients to interact with the game through hand gestures during rehabilitation training. Experimental results show that the system achieves 100% accuracy at an optimal recognition distance of 25 cm. However, accuracy decreases for complex gestures when the distance increases to 45 cm or in low-light conditions, due to the impact of image quality. This system offers a low-cost, immersive rehabilitation solution for stroke patients and provides insights into the development of intelligent rehabilitation technologies.

Many factors can influence how we touch surfaces, where we adapt our finger exploration strategy to the task, material properties, and texture. We studied the influence of finger movement direction, i.e. anteroposterior or lateral, on friction-induced vibrations during touch and perceptual discrimination differences of spatial periods. Three experiments were carried out: two investigating the vibrations induced in the finger, during different sliding velocities and also in changing the investigated textile surfaces, finger angle, velocity, and normal force, as well as a third experiment investigating the perceptual discrimination of textured surfaces. We found that finger vibrations during moving touch were consistently larger in the anteroposterior dimension, especially for frequencies higher than 100 Hz, regardless of the condition or surface. However, for frequencies below 60 Hz, finger vibrations were smaller, similar or greater in the anteroposterior dimension depending on finger velocity, where the velocity equality threshold for movement direction effect was approximately 40 mm/s. In terms of perception, small differences in spatial periods, i.e. 100 or 200 µm, were better discriminated using lateral movements, while discrimination was significantly better using anteroposterior movements for larger spatial period differences, i.e. 300 µm. These results can be explained by different psychometric curves for the movement direction, representing the response level relative to spatial period differences, where the slopes of the discrimination threshold differ between the anteroposterior and lateral directions. These results are discussed relative to the different vibrational behaviours in anteroposterior and lateral movements due to the influence of fingerprints and arm biomechanical behaviour.

Innovative methods for finger gesture recognition have been an active research area, with surface electromyography (sEMG) emerging as a promising approach in human-machine interface applications, especially when visual imaging is impractical. However, sEMG-based gesture recognition is highly susceptible to movement artifacts, individual muscle activation, and changes in hand position, making dynamic gesture recognition challenging. While progress has been made in sEMG data collection and analysis, most studies focus on controlled, static hand positions, limiting real-world applicability. This study integrates a soft wearable sEMG sensor, a Video-Vision-Transform model, and motion sensor-based training to predict finger joint angles and recognize gestures across both static and dynamic hand positions. Despite inter-subject variability, results demonstrate differentiation of finger angles and gestures. For the highly performing subject, recognition accuracy reached 0.85 for static and 0.87 for dynamic settings. This work advances sEMG-based gesture recognition, indicating stable performance across tested static and dynamic conditions, suggesting potential suitability for natural and real-world applications.

To improve the accuracy of Japanese finger-spelled character recognition using an RGB camera, we focused on feature design and refinement of the recognition method. By leveraging angular features extracted via MediaPipe, we proposed a method that effectively captures subtle motion differences while minimizing the influence of background and surrounding individuals. We constructed a large-scale dataset that includes not only the basic 50 Japanese syllables but also those with diacritical marks, such as voiced sounds (e.g., “ga”, “za”, “da”) and semi-voiced sounds (e.g., “pa”, “pi”, “pu”), to enhance the model’s ability to recognize a wide variety of characters. In addition, the application of a change-point detection algorithm enabled accurate segmentation of sign language motion boundaries, improving word-level recognition performance. These efforts laid the foundation for a highly practical recognition system. However, several challenges remain, including the limited size and diversity of the dataset and the need for further improvements in segmentation accuracy. Future work will focus on enhancing the model’s generalizability by collecting more diverse data from a broader range of participants and incorporating segmentation methods that consider contextual information. Ultimately, the outcomes of this research should contribute to the development of educational support tools and sign language interpretation systems aimed at real-world applications.

The pursuit of innovative methods for finger gesture recognition has been an area of active research, with particular emphasis on surface electromyography (sEMG) due to its potential in human-machine interface (HMI) applications, especially in scenarios where visual imaging is impractical. EMG is a promising approach for gesture recognition, but is highly susceptible to movement artifacts, individual motor variability, and changes in hand position, making gesture recognition challenging during dynamic hand movements. While progress has been made in EMG data collection and analysis, most studies focus on controlled, static hand positions, limiting real-world applicability. This study integrates a soft wearable sEMG sensor, a Video-Vision-Transform model, and motion sensor-based training to enhance finger joint angle prediction and gesture recognition across static and dynamic hand positions. Results show substantial variability among participants, yet demonstrate the ability to reach excellent differentiation of finger angles and gestures. For highly performing participants (N=16), recognition accuracy reached 0.85 for static and 0.87 for dynamic conditions. This work advances EMG-based gesture recognition, supporting more robust, real-world applications in dynamic environments.Clinical relevance- Accurate finger gesture recognition using soft wearable EMG sensors has significant implications for neurorehabilitation and assistive technologies. It may enhance motor function assessment, facilitate personalized rehabilitation, and improve prosthetic control by enabling intuitive human-machine interactions. By addressing user variability and optimizing recognition in dynamic conditions, this study contributes to next-generation wearable neurotechnology for neuromuscular disorders, stroke, and spinal cord injuries.

In this paper, we propose a deep learning model that applies PointNet architecture to improve response performance for the occlusion problem beyond the wrist, which was a problem in previous studies. We also evaluate its performance through experiments to estimate finger angles. Positions, joint angles, and accelerations of the fingers are measured and utilized on immersive devices and non-contact interfaces. However, it is difficult to measure them in situations where the wrist is hidden. Therefore, we devise finger angle estimation method that uses point cloud data of forearms as input data. From the results of experiments for 22 participants, the average RMSE was 22.79 and median of R2 was 0.35 when the estimation was performed using a trained model. It suggests that the proposed model can estimate finger angles from the three-dimensional shape of forearms. Moreover, the time required for processing one estimation was 3.798 ms, which indicates that the response performance was good enough.

We present a numerical scheme that solves for the self-similar viscous fingers that emerge from the Saffman–Taylor instability in a divergent wedge. This is based on the formulation by Ben Amar (1991, Phys. Rev. A, vol. 44, pp. 3673–3685). It is demonstrated that there exists a countably infinite set of selected solutions, each with an associated relative finger angle, and furthermore, solutions can be characterised by the number of ripples located at the tip of their finger profiles. Our numerical scheme allows us to observe these ripples and measure them, demonstrating that the amplitudes are exponentially small in terms of the surface tension; the selection mechanism is driven by these exponentially small contributions. A recently published paper derived the selection mechanism for this problem using exponential asymptotic analytical techniques, and obtained bifurcation diagrams that we compare with our numerical results.

SoftGrasp: Adaptive grasping for dexterous hand based on multimodal imitation learning

Early diagnosis of psychomotor diseases such as Parkinson's requires timely and effective medical care, which is often expensive and resource‐intensive. This study proposes a remote‐control system for assisting medical care related to hand movement. Human hand motion is captured using a comfortable, wearable sensory glove, while actuation is achieved via a fabric‐based pneumatic system that drives finger bending. Finite element modeling is conducted to examine how the ratio of the stiff to soft sheet's Young's modulus affects actuator performance, showing that increased ratios lead to greater bending angles. A machine learning model is developed to relate finger angle to actuator pressure. For remote operation, data from the glove are transmitted—physically or virtually—to a separate system, where a medical professional controls the actuator using MATLAB‐based algorithms. This teleoperation method for healthcare is relatively unexplored in current literature. In addition to medical applications such as rehabilitation or Parkinson's monitoring, the system offers the potential for reducing human risk in hazardous settings—such as operating heavy industrial machinery, handling high‐risk lab chemicals, or performing maintenance in contaminated environments.

The biomechanical evaluation of the finger joint angle (FJA) is a fundamental aspect in medical diagnosis and neuromuscular rehabilitation, with direct implications for planning therapeutic strategies and optimizing functional recovery. Currently, FJA quantification methods range from conventional techniques, such as goniometer measurements, to advanced approaches based on Bragg grating fibre-optic strain sensors (FBG) and inertial measurement units (IMU). This study proposes an innovative computational geometric methodology for estimating the flexion and extension angles of finger joints, utilizing IMU sensors integrated into a hardware system based on the ESP32 microcontroller, capable of transmitting real-time data to a dedicated system. A MATLAB graphical user interface (GUI) is used for visualizing and interpreting relevant kin-ematic parameters. Experimental results analysis revealed a maximum approximation error of approximately 3% after implementing a rigorous calibration procedure, using a classical reference method. These findings demonstrate the feasibility of integrating the proposed method into a broader clinical framework for objective monitoring of patient progress in functional rehabilitation programs. The study opens new perspectives for the development of advanced data processing algorithms, including the integration of deep learning neural networks for modelling and optimizing joint movements.

The little finger permanent abduction is an annoying deformity usually along with Wartenberg's sign (a consequence of ulnar nerve palsy), but there are several ways to correct this condition in rheumatoid arthritis as well. We aimed to investigate the effect of surgical intervention on patients. The current study was a clinical trial which was done at Hazrat Fatima Hospital of Tehran, Iran from 2020-2022, where 15 patients with an age range of 21-48 years were investigated. All these patients had complications of ulna nerve damage. After the intervention (tendon transfer), the angle change of the fifth finger was compared with before the surgery. A significant improvement in reducing the angle of the fifth finger after surgery was observed in patients (P<0.05). Tendon transfer is one of the best techniques for the treatment of ulnar injuries.

Abstract To address the complexity of traditional manipulator structures, a bionic five-finger manipulator has been designed and developed. After introducing the manipulator’s structure and functionality, the kinematic model was established using the D-H parameter method, followed by the analysis of forward and inverse kinematics. The fingertip pose was derived using the homogeneous transformation formula, and the accuracy of the theoretical analysis was validated by comparing the mathematical and MATLAB simulation results. The Monte Carlo method was employed to determine the accessible motion space of the mechanical fingertip. Subsequently, trajectory planning for the manipulator was performed based on known initial and end finger angles. A comparison with ADAMS simulation results demonstrated the rationality of the mechanical finger’s movement space, stable speed, and compliance with actual grasping requirements. This study lays a theoretical foundation for the subsequent parameter optimization of the manipulator.

Most traditional rigid grippers can cause damage to the surface of objects in actual production processes and are susceptible to factors such as different shapes, sizes, materials, and positions of the product. This article studies a flexible finger for flexible grippers, more commonly described as PneuNet, designs the structure of the finger, discusses the processing and manufacturing methods of the flexible finger, and prepares a physical model. The influence of structural parameters such as the thickness of the flexible finger and the angle of the air chamber on the bending performance of the finger was analyzed using the Abaqus simulation tool. An RBF-PID control algorithm was used to stabilize the internal air pressure of the flexible fingers. A flexible finger stabilization experimental platform was built to test the ultimate pressure, ultimate bending angle, and end contact force of the fingers, and the simulation results were experimentally verified. The results show that when the thickness of the flexible finger is 2 mm and the air chamber angle is 0 deg, the maximum bending angle of the flexible finger can reach about 136.3°. Under the same air pressure, the bending angle is inversely correlated with the air chamber angle and finger thickness. The experimental error of the bending angle does not exceed 3%, which is consistent with the simulation results as a whole. When the thickness is 2 mm, the maximum end contact force can reach about 1.32 N, and the end contact force decreases with the increase in the air chamber angle. The RBF-PID control algorithm used has improved response speed and a better control effect compared to traditional PID control algorithms. This article provides a clear reference for the application of flexible fingers and flexible grippers, and this research method can be applied to the analysis and design optimization of other soft brakes.

To analyze the structural characteristics of a human hand, data collection gloves were worn for typical grasping tasks. The hand manipulation characteristics, finger end pressure, and finger joint bending angle were obtained via an experiment based on the Feix grasping spectrum. Twelve types of tendon rope transmission paths were designed under the N + 1 type tendon drive mode, and the motion performance of these 12 types of paths applied to tendon-driven fingers was evaluated based on the evaluation metric. The experiment shows that the designed tendon path (d) has a good control effect on the fluctuations of tendon tension (within 0.25 N), the tendon path (e) has the best control effect on the joint angle of the tendon-driven finger, and the tendon path (l) has the best effect on reducing the friction between the tendon and the pulley. The obtained tendon-driven finger motion performance model based on 12 types of tendon paths is a good reference value for subsequent tendon-driven finger structure design and control strategies.

We study self-similar viscous fingering for the case of divergent flow within a wedge-shaped Hele-Shaw cell. Previous authors have conjectured the existence of a countably infinite number of selected solutions, each distinguished by a different value of the relative finger angle. Interestingly, the associated solution branches have been posited to merge and disappear in pairs as the surface tension decreases. For the first time, we demonstrate how the selection mechanism can be derived based on exponential asymptotics. Asymptotic predictions of the finger-to-wedge angle are additionally given for different sized wedges and surface-tension values. The merging of solution branches is explained; this feature is qualitatively different to the case of classic Saffman–Taylor viscous fingering in a parallel channel configuration. Moreover, because the asymptotic framework does not highly depend on specifics of the wedge geometry, the proposed theory for branch merging in our self-similar problem likely relates much more widely to tip-splitting instabilities in time-dependent flows in circular and other geometries, where the viscous fingers destabilise and divide in two.

Today, touchscreens stand as the most prevalent input devices of mobile computing devices (smartphones, tablets, smartwatches). Yet, compared with desktop or laptop computers, the limited shortcut keys and physical buttons on touchscreen devices, coupled with the fat finger problem, often lead to slower and more error-prone input and navigation, especially when dealing with text editing and other complex interaction tasks. We introduce an innovative gesture set based on finger rotations in the yaw, pitch, and roll directions on a touchscreen, diverging significantly from traditional two-dimensional interactions and promising to expand the gesture library. Despite active research in estimation of finger angles, however, the previous work faces substantial challenges, including significant estimation errors and unstable sequential outputs. Variability in user behavior further complicates the isolation of movements to a single rotational axis, leading to accidental disturbances and screen coordinate shifts that interfere with the existing sliding gestures. Consequently, the direct application of finger angle estimation algorithms for recognizing three-dimensional rotational gestures is impractical. SwivelTouch leverages the analysis of finger movement characteristics on the touchscreen captured through original capacitive image sequences, which aims to rapidly and accurately identify these advanced 3D gestures, clearly differentiating them from conventional touch interactions like tapping and sliding, thus enhancing user interaction with touch devices and meanwhile compatible with existing 2D gestures. User study further confirms that the implementation of SwivelTouch significantly enhances the efficiency of text editing on smartphones.

Aiming to address the extended diagnosis and recovery period along with low efficiency in traditional stroke patient rehabilitation, this study introduces a speech recognition-based finger rehabilitation training control system. This system enables patients to perform finger exercises while providing feedback on finger angle, speed, and position. Furthermore, it offers rehabilitation physicians valuable data for evaluation and reference in finger rehabilitation. The system is divided into hardware circuitry, a lower computer control system, and a voice recognition human-computer interaction system, all working in conjunction with the finger movement perception system. By applying the Hidden Markov Model (HMM) algorithm to the voice interaction system for pattern matching, the finger rehabilitation system undergoes simulation testing. The results demonstrate that the proposed rehabilitation training system based on speech recognition meets design requirements, ensuring safety, reliability, and substantial application potential in future finger rehabilitation training.

This research introduces a novel speech recognition-based control system for finger rehabilitation aimed at addressing the inefficiencies and prolonged recovery periods commonly associated with traditional stroke rehabilitation methods. The system enables stroke patients to engage in finger exercises while simultaneously providing real-time feedback on the finger's angle, speed, and position. This feedback is invaluable for rehabilitation physicians, offering critical data for assessing and refining rehabilitation strategies. The system architecture is composed of three main components: the hardware circuitry, a lower-level computer control system, and a voice recognition-based human-computer interaction system, all integrated with a finger movement perception system. Employing the Hidden Markov Model (HMM) for pattern recognition in the voice interaction component, the system has undergone simulation testing to verify its effectiveness. The findings confirm that the speech recognition-based rehabilitation training system meets all design expectations, providing a safe, reliable, and promising approach for enhancing finger rehabilitation practices.