Network For Recognition Research Articles

Spatial-temporal graph neural ODE networks for skeleton-based action recognition.

In the field of skeleton-based action recognition, accurately recognizing human actions is crucial for applications such as virtual reality and motion analysis. However, this task faces challenges such intraindividual action differences and long-term temporal dependencies. To address these challenges, we propose an innovative model called spatial-temporal graph neural ordinary differential equations (STG-NODE). First, in the data preprocessing stage, the dynamic time warping (DTW) algorithm is used to normalize and calculate 3D skeleton data to facilitate the derivation of customized adjacency matrices for improving the influence of intraindividual action differences. Secondly, a custom ordinary differential equation (ODE) integrator is applied based on the initial conditions of the temporal features, producing a solution function that simulates the dynamic evolution trend of the events of interest. Finally, the outstanding ODE solver is used to numerically solve the time features based on the solution function to increase the influence of long-term dependencies on the recognition accuracy of the model and provide it with a more powerful temporal modeling ability. Through extensive experiments conducted on the NTU RGB+D 60 and Kinetics Skeleton 400 benchmark datasets, we demonstrate the superior performance of STG-NODE in the action recognition domain. The success of the STG-NODE model also provides new ideas and methods for the future development of the action recognition field.

F2D-SIFPNet: a frequency 2D Slow-I-Fast-P network for faster compressed video action recognition

F2D-SIFPNet: a frequency 2D Slow-I-Fast-P network for faster compressed video action recognition

Cu/MgO-based resistive random access memory for neuromorphic applications

Resistive Random Access Memory (ReRAM) is considered to be a suitable candidate for future memories due to its low operating voltage, fast access speed, and the potential to be scaled down to nanometer range for ultra-high-density storage. In addition, its ability to retain multi-level resistance states makes it suitable for neuromorphic computing applications. In this paper, we report the resistive switching performance of Cu/MgO/Pt ReRAM. Repetitive resistive switching transitions with low switching voltages (around 1 V), 102 storage windows, and multi-level memory capabilities have been obtained. Biological synaptic plasticity behavior, such as long-duration potentiation/depression and paired-pulse facilitation, has been realized by the Cu/MgO/Pt ReRAM. The simulation of convolutional neural network for handwritten digit recognition is carried out to evaluate its potential application in neuromorphic systems. Finally, the conduction mechanism of the device is studied, and a resistive switching model based on Cu conducting filaments is proposed according to the dependence of I–V results on temperature and electrode size as well as the element distribution in the device. These findings indicate the potential of Cu/MgO/Pt device as high-performance nonvolatile memories and its utilization in future computer systems and neuromorphic computing.

Spatio-Temporal Behavior Detection in Field Manual Labor Based on Improved SlowFast Architecture

Field manual labor behavior recognition is an important task that applies deep learning algorithms to industrial equipment for capturing and analyzing people’s behavior during field labor. In this study, we propose a field manual labor behavior recognition network based on an enhanced SlowFast architecture. The main work includes the following aspects: first, we constructed a field manual labor behavior dataset containing 433,500 fast-track frames and 8670 key frames based on the captured video data, and labeled it in detail; this includes 9832 labeled frames. This dataset provides a solid foundation for subsequent studies. Second, we improved the slow branch of the SlowFast network by introducing the combined CA (Channel Attention) attention module. Third, we enhanced the fast branch of the SlowFast network by introducing the ACTION hybrid attention module. The experimental results show that the recognition accuracy of the improved SlowFast network model with the integration of the two attention modules increases by 7.08%. This implies that the improved network model can more accurately locate and identify manual labor behavior in the field, providing a more effective method for problem solving.

FGDSNet: A Lightweight Hand Gesture Recognition Network for Human Robot Interaction

FGDSNet: A Lightweight Hand Gesture Recognition Network for Human Robot Interaction

基于改进YOLOv5的自然环境下苹果检测方法

To improve the accuracy of apple fruit recognition, enhance the efficiency of automatic picking robots in orchards, and provide effective visual guidance for the picking robot, a target recognition network model based on improved YOLOv5 is proposed. Firstly, the original apple images collected and the data images obtained by different data enhancement methods are used to establish a dataset of 1,879 images, and the dataset is divided into the training set and the test set under 8:2; then for the problem of low detection accuracy of apple fruits in the natural environment due to the mutual obstruction of apple fruits, this paper modifies the backbone network of YOLOv5 by adding the attention mechanism of the Transformer module, the Neck structure is changed from the original PAFPN to BiFPN that can perform two-way weighted fusion, and the Head structure adds the P2 module for shallow down sampling; finally, the recognition test is performed on the dataset, and a comparative analysis is performed according to different evaluation indexes to verify the superiority of the proposed model. The experimental results show that: compared with other existing models and the single-structure improved YOLOv5 model, the comprehensive improved model proposed in this paper has higher detection accuracy, resulting in an increase of 3.7% in accuracy.

SARN: Script-Aware Recognition Network for scene multilingual text recognition

Recent research in script-specific text recognition has achieved remarkable performance on scene text images. However, the challenges of multilingual text in real-world images have not been effectively addressed, thereby limiting the practical application. For multilingual text recognition, the prevailing single-head scheme that uses the same recognition head for all scripts suffers from the issues of enormous alphabet size and linguistic knowledge discrepancies. Meanwhile, the multi-head scheme, which assigns distinct heads for individual scripts, encounters error accumulation within the two-step inference process. In this paper, we explore the influence of script attribute information within existing methods on achieving accurate recognition. Correspondingly, we propose a script-aware recognition network encompassing script identification and text recognition, which leverages script information to make character features more discriminative in text recognition. In particular, we design the script-aware module to utilize script information at the local level, global level, and their combination, respectively. Furthermore, our method is extended to the multi-head scheme, utilizing mutual assistance between script and character features to improve the performance in both script identification and text recognition tasks. Extensive experiments demonstrate that our method obtains superior results in multilingual text recognition, and reveal the importance of script information in alleviating the issues of single-head and multi-head recognition scheme.

Information Extraction from Product Labels: A Machine Vision Approach

This research tackles the challenge of manual data extraction from product labels by employing a blend of computer vision and Natural Language Processing (NLP). We introduce an enhanced model that combines Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) in a Convolutional Recurrent Neural Network (CRNN) for reliable text recognition. Our model is further refined by incorporating the Tesseract OCR engine, enhancing its applicability in Optical Character Recognition (OCR) tasks. The methodology is augmented by NLP techniques and extended through the Open Food Facts API (Application Programming Interface) for database population and text-only label prediction. The CRNN model is trained on encoded labels and evaluated for accuracy on a dedicated test set. Importantly, our approach enables visually impaired individuals to access essential information on product labels, such as directions and ingredients. Overall, the study highlights the efficacy of deep learning and OCR in automating label extraction and recognition.

AC-CfC: An attention-based convolutional closed-form continuous-time neural network for raw multi-channel EEG-based emotion recognition

Emotion recognition based on electroencephalogram (EEG) is a critical task in the field of affective brain-computer interfaces. However, due to the non-stationarity and individual variability of EEG, hand-designed features cannot adequately capture the nonlinear and high-dimensional properties of raw EEG. Currently, spatial–temporal models have been verified to effectively capture the spatial–temporal information for EEG. However, these models are characterized by complex structures, large parameter counts, and the need for extensive training data. To overcome these disadvantages, this paper propose an end-to-end spatial–temporal model for emotion recognition based on raw EEG, called attention-based convolutional closed-form continuous-time neural network (AC-CfC). This model employs a channel attention mechanism to weight and encode EEG, capturing channel dependencies. Subsequently, one-dimensional convolutional neural networks and closed-form continuous-time neural networks are used to extract deep spatial–temporal features. Additionally, an adaptive loss-controlling mechanism is designed to enhance the model's decision-making ability between classes that are easily confused. To verify the effectiveness of the proposed model, experiments are conducted on the DEAP and DREAMER datasets. The average accuracies of the proposed model reach to 94.76% and 93.01% for valence and arousal in subject-independent experiments on the DEAP dataset, with an improvement of 11.8% and 8.73% respectively compared with the state-of-art model ACRNN. On the DREAMER dataset, the average accuracies reach to 81.83%, 81.22%, and 80.63% for valence, arousal and dominance, with an improvement of 2.55%, 6.64%, and 6.98% respectively over ACRNN. These results show that our proposed model exhibits better performance than some state-of-art models in the same category.

Integrated Sensor Fusion and Multi-Modal Hierarchical Neural Network for Activity Recognition in Lower Limb Prosthetics

Activity recognition plays pivotal role in enhancing functionality for prosthetic devices, ensuring seamless integration with users' movements. However, the complexity arises from the diverse data sources, including acceleration, angular velocity, joint angles, orientation, electromyography (EMG), and marker data, necessitating a robust approach to overcome challenges in information integration. The primary challenge lies in the effective utilization of multiple sensor modalities, each with unique characteristics and potential noise sources. The proposed solution addresses this by employing advanced sensor fusion techniques, such as Kalman filtering, during data collection. Synchronization and resampling ensure temporal consistency, while noise reduction techniques, such as low-pass filters, mitigate signal distortions. To further refine the process, a hybrid optimization-based feature selection Adaptive Step Size in Marine Predators Algorithm (ASSMPA) is introduced, focusing on marker data features. ASSMPA synergizes Marine Predators Algorithm (MPA) and Pathfinder Algorithm (PFA) for optimal feature selection in marine predator pathfinding tasks. The Feature Fusion step integrates attention mechanisms to dynamically weigh the significance of different sensor modalities during the fusion process. This strategic fusion enhances the overall performance of the Multi-Modal Hierarchical Neural Network (MMHNN). The proposed model is implemented using Python

Multipath Attention and Adaptive Gating Network for Video Action Recognition

3D CNN networks can model existing large action recognition datasets well in temporal modeling and have made extremely great progress in the field of RGB-based video action recognition. However, the previous 3D CNN models also face many troubles. For video feature extraction convolutional kernels are often designed and fixed in each layer of the network, which may not be suitable for the diversity of data in action recognition tasks. In this paper, a new model called Multipath Attention and Adaptive Gating Network (MAAGN) is proposed. The core idea of MAAGN is to use the spatial difference module (SDM) and the multi-angle temporal attention module (MTAM) in parallel at each layer of the multipath network to obtain spatial and temporal features, respectively, and then dynamically fuses the spatial-temporal features by the adaptive gating module (AGM). SDM explores the action video spatial domain using difference operators based on the attention mechanism, while MTAM tends to explore the action video temporal domain in terms of both global timing and local timing. AGM is built on an adaptive gate unit, the value of which is determined by the input of each layer, and it is unique in each layer, dynamically fusing the spatial and temporal features in the paths of each layer in the multipath network. We construct the temporal network MAAGN, which has a competitive or better performance than state-of-the-art methods in video action recognition, and we provide exhaustive experiments on several large datasets to demonstrate the effectiveness of our approach.

Adaptive fuzzy energy management strategy for range‐extended electric vehicles integrated with deep learning

AbstractA novel framework for adaptive energy management, rooted in deep learning principles, is proposed to minimize fuel consumption in extended‐range electric vehicles amidst intricate driving scenarios. This innovative approach integrates a long short‐term memory (LSTM) network for pattern recognition across three driving patterns and an adaptive fuzzy controller. To mitigate the impact of poor hyperparameter selection on recognition accuracy, Gray Wolf Optimization is employed to optimize the hidden layer nodes, training times, and learning rate of the LSTM. Simultaneously, a genetic algorithm is utilized to optimize the vertex coordinates of the fuzzy control membership function, enabling the adaptive adjustment of parameters in the fuzzy energy management strategy. The condition recognition model accurately identifies the vehicle's driving status and seamlessly transitions to an energy management strategy tailored to the present conditions. This ensures optimal operation, enhancing overall fuel efficiency and performance. The simulation results robustly validate the efficacy of this approach: the GWO‐LSTM network achieves an impressive 97.7% accuracy in recognizing working conditions, surpassing the 88.9% accuracy of the traditional LSTM network. Furthermore, the fuel consumption reduction achieved by the adaptive fuzzy energy management strategy amounts to 11.9% compared with the conventional fuzzy energy management approach. This outcome underscores the tangible enhancement in vehicle fuel economy resulting from the seamless integration of deep learning techniques.

Paired relation feature network for spatial relation recognition

Recognizing relations between objects in an image is challenging for neural networks because some relations may not have obvious dedicated visual features. This paper proposes a Paired Relation Feature Network (PRFN), where all spatial and semantic features are extracted from the subject–object pair jointly, without using any hand-crafted features. PRFN includes a paired 2D spatial feature module that can learn the representative features from a pair of bounding boxes. By focusing on the paired depth feature between the subject and object, the problem of depth feature extraction is simplified to the recognition of a ternary relation {−1, 0, 1}, which is much easier to learn from training data. Experimental results demonstrate the effectiveness of PRFN for both the cases of RGB-D images and RGB images with estimated depth.

Multi-scale aware turbulence network for underwater object recognition

Underwater imagery is subject to distortion, and the presence of turbulence in the fluid medium poses difficulties in accurately discerning objects. To tackle these challenges pertaining to feature extraction, this research paper presents a novel approach called the multi-scale aware turbulence network (MATNet) method for underwater object identification. More specifically, the paper introduces a module known as the multi-scale feature extraction pyramid network module, which incorporates dense linking strategies and position learning strategies to preprocess object contour features and texture features. This module facilitates the efficient extraction of multi-scale features, thereby enhancing the effectiveness of the identification process. Following that, the extracted features undergo refinement through comparison with positive and negative samples. Ultimately, the study introduces multi-scale object recognition techniques and establishes a multi-scale object recognition network for the precise identification of underwater objects, utilizing the enhanced multi-scale features. This process entails rectifying the distorted image and subsequently recognizing the rectified object. Extensive experiments conducted on an underwater distorted image enhancement dataset demonstrate that the proposed method surpasses state-of-the-art approaches in both qualitative and quantitative evaluations.

Unleashing the power of Convolutional Neural Networks in license plate recognition and beyond

This article explores the application of Convolutional Neural Networks (CNN) in the field of license plate recognition. It begins by introducing the architecture of CNN, which consists of three key layers: Convolutional Layers, Pooling Layers, and Fully Connected Layers. The article then references three relevant papers that demonstrate how CNNs are applied in license plate recognition. The first paper utilizes TensorFlow to construct a CNN model and integrates it with an STM32MP157 embedded chip for license plate recognition. The second paper presents a real-time car license plate detection and recognition method called Multi-Task Light CNN, emphasizing robustness. The third paper employs the ResNet+FPN feature extraction network of the Mask R-CNN model and annotates a license plate dataset. The article highlights the promising future of CNNs in various fields beyond license plate recognition, emphasizing their potential for further development and industrial applications. CNNs have proven to be versatile and powerful tools in computer vision, offering solutions to a wide range of problems. Their adaptability and effectiveness make them a key player in the ongoing advancement of artificial intelligence and automation technologies.

DC-BVM: Dual-channel information fusion network based on voting mechanism

Emotion recognition in conversations (ERC) has been challenging due to the dynamics and complexity of emotions in conversations. Most current emotion recognition studies have focused on modeling temporal dimensions, such as context-sensitive dependencies while ignoring the spatial dimensional relationships between discourses. In this paper, we propose a dual-channel information fusion network based on a voting mechanism, in which we model modal information in two dimensions: distance dependence is constructed in the spatial dimension, and context dependence is constructed in the temporal dimension. We aim to extract more comprehensive and accurate sentiment features to recognize verbal emotions from limited data. In addition, we discard the traditional modal fusion method and propose a fusion strategy based on the voting mechanism, which significantly accelerates the model convergence speed and recognition performance. We conducted experiments on two benchmark datasets, IEMOCAP and MELD, respectively, and the experimental results demonstrate the superiority of the designed network in emotion recognition.

Dual perspective denoising model for session-based recommendation

Session-based recommendation predicts the next interaction option for an anonymous user based on his/her short-term behavior records. A key challenge faced by session-based recommendation is the processing of noise information. That is, the precise identification of noise items and noise connections. Existing research typically follows two assumptions: (1) the latest interaction item is directly treated as the user’s real-time preference; (2) the temporal order relationship between items is directly treated as the user’s true interaction intention conversion pattern. These strict assumptions ignore two facts: (1) the items that the user accidentally clicked on also exist in the interaction session. (2) the connection edges caused by noise items or user interaction intention drift are also fused into the explicit sequential structure. To address the above two issues, we propose a new Dual Perspective Denoising Model for session-based recommendation called DPDM. Specifically, DPDM mainly consists of two parts: Noise Item Recognition Network and Noise Structure Reconstruction Network. The Noise Item Recognition Network removes noise items from the session through a sparse attention mechanism and virtual target representation. The Noise Structure Reconstruction Network composed of Sequence Module and Co-occurrence Module is used to mine users’ real interaction intention conversion patterns. The Sequence Module captures sequence relationships after removing noise items. The Co-occurrence Module eliminates noise connections by optimizing explicit interaction sequences with L0 activation regularization. Extensive experiments were conducted on three representative real-world datasets, and DPDM achieved positive results, demonstrating the effectiveness of the components in the model.

Multimodal Visual-Semantic Representations Learning for Scene Text Recognition

Scene Text Recognition (STR), the critical step in OCR systems, has attracted much attention in computer vision. Recent research on modeling textual semantics with Language Model (LM) has witnessed remarkable progress. However, LM only optimizes the joint probability of the estimated characters generated from the Vision Model (VM) in a single language modality, ignoring the visual-semantic relations in different modalities. Thus, LM-based methods can hardly generalize well to some challenging conditions, in which the text has weak or multiple semantics, arbitrary shape, and so on. To migrate the above issue, in this paper, we propose Multimodal Visual-Semantic Representations Learning for Text Recognition Network (MVSTRN) to reason and combine the multimodal visual-semantic information for accurate Scene Text Recognition. Specifically, our MVSTRN builds a bridge between vision and language through its unified architecture and has the ability to reason visual semantics by guiding the network to reconstruct the original image from the latent text representation, breaking the structural gap between vision and language. Finally, the tailored multimodal Fusion (MMF) module is motivated to combine the multimodal visual and textual semantics from VM and LM to make the final predictions. Extensive experiments demonstrate our MVSTRN achieves state-of-the-art performance on several benchmarks.

Deep neural networks for automatic speaker recognition do not learn supra-segmental temporal features

While deep neural networks have shown impressive results in automatic speaker recognition and related tasks, it is dissatisfactory how little is understood about what exactly is responsible for these results. Part of the success has been attributed in prior work to their capability to model supra-segmental temporal information (SST), i.e., learn rhythmic-prosodic characteristics of speech in addition to spectral features. In this paper, we (i) present and apply a novel test to quantify to what extent the performance of state-of-the-art neural networks for speaker recognition can be explained by modeling SST; and (ii) present several means to force respective nets to focus more on SST and evaluate their merits. We find that a variety of CNN- and RNN-based neural network architectures for speaker recognition do not model SST to any sufficient degree, even when forced. The results provide a highly relevant basis for impactful future research into better exploitation of the full speech signal and give insights into the inner workings of such networks, enhancing explainability of deep learning for speech technologies.

Discriminative Segment Focus Network for Fine-grained Video Action Recognition

Fine-grained video action recognition aims to identify minor and discriminative variations among fine categories of actions. While many recent action recognition methods have been proposed to better model spatio-temporal representations, how to model the interactions among discriminative atomic actions to effectively characterize inter-class and intra-class variations has been neglected, which is vital for understanding fine-grained actions. In this work, we devise a Discriminative Segment Focus Network (DSFNet) to mine the discriminability of segment correlations and localize discriminative action-relevant segments for fine-grained video action recognition. Firstly, we propose a hierarchic correlation reasoning (HCR) module which explicitly establishes correlations between different segments at multiple temporal scales and enhances each segment by exploiting the correlations with other segments. Secondly, a discriminative segment focus (DSF) module is devised to localize the most action-relevant segments from the enhanced representations of HCR by enforcing the consistency between the discriminability and the classification confidence of a given segment with a consistency constraint. Finally, these localized segment representations are combined with the global action representation of the whole video for boosting final recognition. Extensive experimental results on two fine-grained action recognition datasets, i.e. , FineGym and Diving48, and two action recognition datasets, i.e. , Kinetics400 and Something-Something, demonstrate the effectiveness of our approach compared with the state-of-the-art methods.