Continuous Recognition Research Articles

Ensembled Pretrained Model-Based Adaptive Sequential Trace Recognition for Moving Objects

In this paper, we propose an adaptive continuous trajectory recognition method for badminton sports scenarios. First, we adopt an integrated pre-training model with high generality and expressive ability as the infrastructure. By pre-training large-scale badminton game data, the model can learn the features of badminton sports from rich and diverse trajectory data. Second, to adapt to the trajectory differences in different scenarios, we introduce an adaptive mechanism. By extracting and encoding features from real-time acquired badminton motion trajectories, the model can adjust its own parameters and weight assignments according to the changes in the current scene to optimally recognize and track the motion targets. The algorithm also employs the Kalman filter aggregation Enhanced Correlation Coefficient (ECC) method of the motion model to improve the prediction accuracy. Finally, a series of experiments and comparisons are conducted to validate the effectiveness of the approach. The results show that our proposed adaptive continuous trajectory recognition method for badminton achieves better performance in different scenarios and various complexities, and has higher accuracy and robustness than the traditional method. The FDA-SSD model operates at 28.7[Formula: see text]fps, which is about 16.5% faster than the traditional SSD. In the target tracking experiments, the target tracking mean squared error is about 1.0%. In the target tracking experiments based on the FDA and geometrically constrained localization method, the root-mean-square error of the target tracking is less than 4.72[Formula: see text]cm.

Continuous sign language recognition enhanced by dynamic attention and maximum backtracking probability decoding

Continuous sign language recognition enhanced by dynamic attention and maximum backtracking probability decoding

A Two-Phase and Bi-Level Spatial Configuration Methodology of Shelters Based on a Circular Assignment Model and Evacuation Traffic Flow Allocation

With the continued recognition of the devastating effects of natural hazards, the construction of shelters has become essential in urban disaster preparedness planning systems. After analyzing the deficiency of the conventional spatial allocation model of shelters and the hierarchy of evacuation assignments, this study proposes a bi-level and two-phase spatial configuration methodology of shelters. The first hierarchy aims to evacuate refugees from demand blocks to both emergency shelters and resident emergency congregate shelters. The second hierarchy aims to transfer refugees from selected shelters in the first hierarchy to resident emergency congregate shelters. Each hierarchy contains two phases of optimizing calculations. The optimization objects for the first phase and second phase are minimizing the number of new shelters and the evacuation time, respectively. A genetic algorithm and exhaustive approach are programmed to determine the solution of the model in the first and second phases, respectively. The evacuation assignment rule is proposed based on the gravity model, which distributes evacuees proportionally to nearby shelters. This study uses the deterministic user equilibrium problem to present the evacuation traffic flow allocation, which improves the scientificity of the location model of shelters. The refuge demands differentiate the population between daytime and nighttime through mobile signaling data and improve the accuracy from the plot scale to the building scale. Using mobile signaling data to differentiate refuge demands between day and night populations enhances the model’s precision. Finally, to validate the proposed methodology, this study selected the main area of Changshu City, Jiangsu Province, China, which has a population of 1.6 million, as a case study area.

Classification Strategies for Radar-Based Continuous Human Activity Recognition With Multiple Inputs and Multilabel Output

Classification Strategies for Radar-Based Continuous Human Activity Recognition With Multiple Inputs and Multilabel Output

Occlusion Resistant Spatio-Temporal Hybrid Cue Network for Indian Sign Language Recognition and Translation

Objective: To tackle the issues of occlusion in human skeleton extraction and simplify the pixel matching related to the human skeleton structure for efficient Indian Sign Language (ISL) recognition and translation. Methods: This paper presents Occlusion-Resistant STHCN (OSTHCN) to tackle the occlusion problem in human skeleton extraction for effective ISL recognition and translation. This model incorporates Skeleton Occupancy Likelihood Map estimation using B-Spline curves to enhance the skeleton extraction. Due to occlusions caused by fingers and hands, the extracted skeleton is composed of disconnected skeletal subgraphs. Consequently, each observed skeleton is represented as a probability distribution along an ellipsoidal contour, originating from the central points of the skeleton. A heuristic technique estimates occluded skeletons using 3D probability map with an occupancy grid where each voxel indicates skeleton likelihood. The occupancy distribution is updated using observed branch clusters across image sequences, detecting occluded skeletons by finding minimum-cost paths. Finally, Maximally connected subgraphs are merged into a main graph by finding minimum-cost paths in the 3D likelihood map, enabling the prediction of occluded skeleton parts for ISL recognition and translation. Findings: OSTHCN model achieved an accuracy of 96.74% on the ISL Continuous Sign Language Translation Recognition (ISL-CSLTR) dataset outperforming existing prediction models. Novelty: This model employs a unique occlusion-handling strategy for skeleton extraction, estimating occluded part, integrating connected subgraphs via minimal cost path searches for more precise skeleton parts and enhancing accuracy for ISL recognition and translation. Keywords: Sign Language, Graph Convolutional Neural Network, Ellipsoidal Contour, 3D Likelihood Map, Occupancy Probability

Construction of tennis pose estimation and action recognition model based on improved ST-GCN

With the rapid growth of computer vision and deep learning technologies, the application of pose estimation and action recognition in sports training has become increasingly widespread. Due to factors such as complex movements, fast speed, and limb occlusion, pose estimation and action recognition in tennis face significant challenges. Therefore, this study first introduces selective dropout and pyramid region of interest pooling layer strategies in fast region convolutional neural networks. Secondly, a pose estimation algorithm based on multi-scale fusion pose residual network 50 is designed, and finally a spatiotemporal graph convolutional network model is constructed by fusing channel attention module and multi-scale dilated convolution module. The data showed that the average detection accuracy of the improved attitude residual network 50 was 70.4%, and the accuracy of object detection for small, medium, and large objects was 57.4%, 69.3%, and 79.2%, respectively. The continuous action recognition accuracy and inter action fluency detection time of the improved spatiotemporal graph convolutional network were 93.8% and 19.2 ms, respectively. When the sample size was 1000, its memory usage was 1378 MB and the running time was 32.7 ms. Experiments have shown that the improved model achieves high accuracy and robustness in tennis action recognition tasks, especially in complex scenes and limb occlusion conditions, where the model shows significant advantages. This study aims to provide an efficient and accurate motion recognition technology for tennis posture analysis and intelligent training.

Machine Learning and Deep Learning Approaches for Arabic Sign Language Recognition: A Decade Systematic Literature Review.

Sign language (SL) is a means of communication that is used to bridge the gap between the deaf, hearing-impaired, and others. For Arabic speakers who are hard of hearing or deaf, Arabic Sign Language (ArSL) is a form of nonverbal communication. The development of effective Arabic sign language recognition (ArSLR) tools helps facilitate this communication, especially for people who are not familiar with ArSLR. Although researchers have investigated various machine learning (ML) and deep learning (DL) methods and techniques that affect the performance of ArSLR systems, a systematic review of these methods is lacking. The objectives of this study are to present a comprehensive overview of research on ArSL recognition and present insights from previous research papers. In this study, a systematic literature review of ArSLR based on ML/DL methods and techniques published between 2014 and 2023 is conducted. Three online databases are used: Web of Science (WoS), IEEE Xplore, and Scopus. Each study has undergone the proper screening processes, which include inclusion and exclusion criteria. Throughout this systematic review, PRISMA guidelines have been appropriately followed and applied. The results of this screening are divided into two parts: analysis of all the datasets utilized in the reviewed papers, underscoring their characteristics and importance, and discussion of the ML/DL techniques' potential and limitations. From the 56 articles included in this study, it was noticed that most of the research papers focus on fingerspelling and isolated word recognition rather than continuous sentence recognition, and the vast majority of them are vision-based approaches. The challenges remaining in the field and future research directions in this area of study are also discussed.

R-ATCN: continuous human activity recognition using FMCW radar with temporal convolutional networks

Abstract The utilization of millimeter-wave radar sensors for continuous human activity recognition technology has garnered significant interest. Prior research predominantly concentrated on recursive neural networks, which often incorporate numerous extraneous information features, hindering the ability to make precise and effective predictions for ongoing activities. In response to this challenge, this paper introduces a dual-dilated one-dimensional temporal convolutional network model with an attention mechanism (R-ATCN). By stacking temporal convolutions to enhance the receptive field without compromising temporal resolution, the R-ATCN effectively captures features. Additionally, the attention mechanism is employed to capture crucial frame information related to activity transitions and overall features. The study gathered 60 data sets from 5 participants utilizing frequency modulated continuous wave radar. It encompassed 8 various activities lasting a total of 52.5 min, with randomized durations and transition times for each activity. To evaluate the performance of the model, this paper also introduces evaluation metrics such as short-time tolerance (STT) score. Experimental results show that the R-ATCN model outperforms other contrastive models in terms of segmental F1-score and STT scores. The effectiveness of the proposed model lies in its ability to accurately identify ongoing human activities within indoor environments.

Spatial Temporal Aggregation for Efficient Continuous Sign Language Recognition

Spatial Temporal Aggregation for Efficient Continuous Sign Language Recognition

Continuous Human Action Recognition by Multiple-Object-Detection-Based FMCW Radar

Continuous Human Action Recognition by Multiple-Object-Detection-Based FMCW Radar

Multimodal hand/finger movement sensing and fuzzy encoding for data‐efficient universal sign language recognition

AbstractWearable sign language recognition helps hearing/speech impaired people communicate with non‐signers. However current technologies still unsatisfy practical uses due to the limitations of sensing and decoding capabilities. Here, A continuous sign language recognition system is proposed with multimodal hand/finger movement sensing and fuzzy encoding, trained with small word‐level samples from one user, but applicable to sentence‐level language recognition for new untrained users, achieving data‐efficient universal recognition. A stretchable fabric strain sensor is developed by printing conductive poly(3,4‐ethylenedioxythiophene):poly(styrenesulfonate) (PEDOT:PSS) ink on a pre‐stretched fabric wrapping rubber band, allowing the strain sensor with superior performances of wide sensing range, high sensitivity, good linearity, fast dynamic response, low hysteresis, and good long‐term reliability. A flexible e‐skin with a homemade micro‐flow sensor array is further developed to accurately capture three‐dimensional hand movements. Benefitting from fabric strain sensors for finger movement sensing, micro‐flow sensor array for 3D hand movement sensing, and human‐inspired fuzzy encoding for semantic comprehension, sign language is captured accurately without the interferences from individual action differences. Experiment results show that the semantic comprehension accuracy reaches 99.7% and 95%, respectively, in recognizing 100 isolated words and 50 sentences for a trained user, and achieves 80% in recognizing 50 sentences for new untrained users.image

Efficient resource recovery from food waste digestate via hydrothermal treatment and its application as organic fertilizer

With the continuous recognition of green, organic and non-polluting products, organic fertilizers play an increasingly vital role in agricultural production. Among them, hydrochar-based organic fertilizer has attracted widespread attention recently. The present study evaluated the potential of digestate from anaerobic digestion of food waste for the preparation of hydrochar-based organic fertilizer by straw-based, FeCl3-catalyzed hydrothermal carbonization (HTC). Under the optimal conditions, a hydrochar-based organic fertilizer with > 25 wt% humus content and limited pollution risk was successfully prepared. The pot experiment demonstrated the feasibility of improving the physicochemical properties of red soil and promoting crop growth after adding hydrochar in place of commercial fertilizer. In addition, the function of zeolite on nutrient recovery in hydrothermal liquid (HTL) was analyzed, and preparing the slow-release organic fertilizer by mixing the nutrient-rich zeolite with hydrochar in a mass ratio of 1:4 was proposed. This work has significant implications for achieving the efficient resource recovery of digestate.

Continuous sign language recognition algorithm based on object detection and variable-length coding sequence

Currently, continuous sign language recognition faces challenges such as difficulty in acquiring skeletal data, long training time for Three-Dimensional convolutional neural networks, and easy occlusion and blurring of hands. To address these problems, this paper proposes a continuous sign language recognition method based on target detection and coding sequence. The algorithm uses Dual-branch Shuffle Attention Mechanis-You Only Look Once version X (DSA-YOLOX) detection network to detect the head and hands, and encodes the sign language video according to the partition to achieve the transformation from Three-Dimensional to One-Dimensional; and then uses the proposed Bi-directional Long Short-Term Memory (BiLSTM) hand coding sequence classification model with jointly weighted Fast Dynamic Time Warping (FastDTW) to extract hand coding similarity and features while reducing the number of parameters to achieve the classification and recognition of unequal-length hand coding sequences. From the results of ablation experiments and comparison experiments, all parts of the improvement perform well. The word error rate (WER) of this paper’s method is reduced by 21.26% compared to Dynamic Time Warping-Hidden Markov Model (DTW-HMM) and 11.53% compared to Long Short-Term Memory-A(LSTM-A); the Giga Floating-point Operations Per Second(GFLOPs) of the algorithm are reduced dramatically, which is about 1/13 of the Visual Alignment Constraint(VAC) model and 1/57 of the Spatial-Temporal Multi-Cue(STMC) model; and the algorithm takes better account of the speed and accuracy of sign language recognition.

A Systematic Review of Accessibility Techniques for Online Platforms: Current Trends and Challenges

Accessibility in online platforms is a critical concern in our increasingly digital world, where information and services are predominantly accessed through the Internet. The purpose of this systematic review is to provide a comprehensive overview of the current state of the art in online accessibility technologies, and it is focused on key tools such as sign language recognition, speech-to-text, text-to-speech, and voice recognition. Despite advancements in digital inclusivity, numerous technical limitations persist, which limit the accessibility of online content for individuals with disabilities. Our findings indicate that while speech and voice technologies have achieved good accuracies and low word error rates, further research is needed to improve the accuracy and usability of sign language recognition systems, especially for continuous sign language recognition, as they have low accuracy. In this review, we analyzed research articles and publications from well-known databases, including Google Scholar, Elsevier, IEEE Xplore, and Springer. In order to ensure a high standard of quality, we applied the PRISMA 2020 and PEDro methodologies to quantitatively and qualitatively filter the thousands of articles provided by these databases, and we selected only studies that were related to our study. Key areas of investigation included the performance and accuracy of sign language interfaces, speech-to-text, text-to-speech, and speech recognition applications and the compatibility of these technologies with different platforms and devices. This review also explores the role of emerging technologies such as artificial intelligence (AI) and machine learning (ML) in enhancing accessibility and personalizing user experiences. Through a critical analysis of current solutions and a discussion of existing gaps, this paper offers insights into potential improvements and future directions for creating more accessible online environments. The findings might be valuable to researchers and developers dedicated to promoting digital inclusivity and equality.

Moral imperatives of modern Russian constitutionalism

The article analyzes the dynamics of moral imperatives of Russian constitutionalism as moral principles or rules, categorical requirements, commandments or norms, according to which the multinational people of Russia build their way of life. The article argues the need to overcome the moral neutrality of public power, enshrined in the Constitution of the Russian Federation in 1993. Particular attention is paid to the change in moral imperatives after the reform of the Constitution of the Russian Federation in 2020. Namely, the love for the Fatherland, based on patriotic education, preservation of the memory of ancestors who passed on to us ideals and faith in God, on the protection of historical truth and all-Russian cultural identity; recognition of continuity in the development of the Russian state; reliance in social development on traditional spiritual and moral values.

Visual context learning based on cross-modal knowledge for continuous sign language recognition

Visual context learning based on cross-modal knowledge for continuous sign language recognition

Dynamical semantic enhancement network for continuous sign language recognition

Dynamical semantic enhancement network for continuous sign language recognition

A Novel Gesture Recognition Network based on LSTM

Surface electromyography (sEMG) is a rich source of physiological data that reflects human movement intentions. The recognition of gestures through sEMG has garnered significant interest in the realms of human-computer interaction and rehabilitation. Currently, the majority of research on gesture recognition utilizing sEMG signals relies on discrete segmentation techniques, often overlooking the nuances of continuous and natural movements. A novel gesture recognition network based on LSTM for recognizing gestures from sEMG signals is prpopsed. The sEMG sensors are strategically positioned based on anatomical and muscular functions to optimize the capture of relevant physiological signals. In this study, finger curvature is employed to characterize gesture states, allowing each gesture at any given moment to be represented by a collection of varying finger curvatures, thereby enabling continuous gesture recognition. The experimental results indicate that this method significantly enhances the ability to mine representations from sEMG signals, offering valuable insights for deep learning models focused on human gesture recognition.

Beyond Granularity: Enhancing Continuous Sign Language Recognition with Granularity-Aware Feature Fusion and Attention Optimization

The advancement of deep learning techniques has significantly propelled the development of the continuous sign language recognition (cSLR) task. However, the spatial feature extraction of sign language videos in the RGB space tends to focus on the overall image information while neglecting the perception of traits at different granularities, such as eye gaze and lip shape, which are more detailed, or posture and gestures, which are more macroscopic. Exploring the efficient fusion of visual information of different granularities is crucial for accurate sign language recognition. In addition, applying a vanilla Transformer to sequence modeling in cSLR exhibits weak performance because specific video frames could interfere with the attention mechanism. These limitations constrain the capability to understand potential semantic characteristics. We introduce a feature fusion method for integrating visual features of disparate granularities and refine the metric of attention to enhance the Transformer’s comprehension of video content. Specifically, we extract CNN feature maps with varying receptive fields and employ a self-attention mechanism to fuse feature maps of different granularities, thereby obtaining multi-scale spatial features of the sign language framework. As for video modeling, we first analyze why the vanilla Transformer failed in cSLR and observe that the magnitude of the feature vectors of video frames could interfere with the distribution of attention weights. Therefore, we utilize the Euclidean distance among vectors to measure the attention weights instead of scaled-dot to enhance dynamic temporal modeling capabilities. Finally, we integrate the two components to construct the model MSF-ET (Multi-Scaled feature Fusion–Euclidean Transformer) for cSLR and train the model end-to-end. We perform experiments on large-scale cSLR benchmarks—PHOENIX-2014 and Chinese Sign Language (CSL)—to validate the effectiveness.

Machine Learning-Assisted Gesture Sensor Made with Graphene/Carbon Nanotubes for Sign Language Recognition.

Gesture sensors are essential to collect human movements for human-computer interfaces, but their application is normally hampered by the difficulties in achieving high sensitivity and an ultrawide response range simultaneously. In this article, inspired by the spider silk structure in nature, a novel gesture sensor with a core-shell structure is proposed. The sensor offers a high gauge factor of up to 340 and a wide response range of 60%. Moreover, the sensor combining with a deep learning technique creates a system for precise gesture recognition. The system demonstrated an impressive 99% accuracy in single gesture recognition tests. Meanwhile, by using the sliding window technology and large language model, a high performance of 97% accuracy is achieved in continuous sentence recognition. In summary, the proposed high-performance sensor significantly improves the sensitivity and response range of the gesture recognition sensor. Meanwhile, the neural network technology is combined to further improve the way of daily communication by sign language users.