Improve Recognition Performance Research Articles

Experimental Study on Using Synthetic Images as a Portion of Training Dataset for Object Recognition in Construction Site

The application of Artificial Intelligence (AI) across various industries necessitates the acquisition of relevant environmental data and the implementation of AI recognition learning based on this data. However, the data available in real-world environments are limited and difficult to obtain. Construction sites represent dynamic and hazardous environments with a significant workforce, making data acquisition challenging and labor-intensive. To address these issues, this experimental study explored the potential of generating synthetic data to overcome the challenges of obtaining data from hazardous construction sites. Additionally, this research investigated the feasibility of hybrid dataset in securing construction-site data by creating synthetic data for scaffolding, which has a high incidence of falls but low object recognition rates due to its linear object characteristics. We generated a dataset by superimposing scaffolding objects, from which the backgrounds were removed, onto various construction site background images. Using this dataset, we produced a hybrid dataset to assess the feasibility of synthetic data for construction sites and to evaluate improvements in object recognition performance. By finding the optimal composition ratio with real data and conducting model training, the highest accuracy was achieved at an 8:2 ratio, with a construction object recognition accuracy of 0.886. Therefore, this study aims to reduce the risk and labor associated with direct data collection at construction sites through a hybrid dataset, achieving data generation at a low cost and high efficiency. By generating synthetic data to find the optimal ratio and constructing a hybrid dataset, this research demonstrates the potential to address the problems of data scarcity and data quality on construction sites. The improvement in recognition accuracy of the construction safety management system is anticipated, suggesting that the creation of synthetic data for constructing a hybrid dataset can reduce construction safety-accident issues.

Resource-efficient, sensor-based human activity recognition with lightweight deep models boosted with attention

With their automatic feature extraction capabilities, deep learning models have become more widespread in sensor-based human activity recognition, particularly on larger datasets. However, their direct use on mobile and wearable devices is challenging due to the extensive resource requirements. Concurrently, attention-based models are emerging to improve recognition performance by dynamically emphasizing relevant parts of features and disregarding the irrelevant ones, particularly in the computer vision domain. This study introduces a novel application of attention mechanisms to smaller deep architectures, investigating whether smaller models can achieve comparable recognition performance to larger models in sensor-based human activity recognition systems while keeping resource usage at lower levels. For this purpose, we integrate the convolutional block attention module into a hybrid model, deep convolutional and long short-term memory network. Experiments are conducted using five public datasets in three model sizes: lightweight, moderate and original. The results show that applying attention to the lightweight model enables achieving similar recognition performances to the moderate-size model, and the lightweight model requires approximately 2–13 times fewer parameters and 3.5 times fewer flops. We also conduct experiments with sensor data at lower sampling rates and from fewer sensors attached to different body parts. The results show that attention improves recognition performance under lower sampling rates, as well as under higher sampling rates when model sizes are smaller, and mitigates the impact of missing data from one or more body parts, making the model more suitable for real-world sensor-based applications.

Double reuses based residual network

Deep residual network (ResNet) had shown remarkable performance in image recognition tasks, due to its shorter connections between layers close to the input and those close to the output. And densely connected convolutional network (DenseNet) had further improved recognition performance by dense feature reuses. To improve the performance of residual units in ResNet and feature reuses in DenseNet, we propose a simple and effective convolutional network architecture, named double reuses based residual network (DRRNet). DRRNet improves the residual unit of ResNet, where the feature reuse connections are added to combine all feature maps from the convolutional layers to produce the residual, and uses a residual reuse path outside units to reuse all residuals as the final feature maps for classification. Residual learning used in DRRNet can alleviate the vanishing-gradient problem. The double reuses including inner-unit feature reuses and outer-unit residual reuses effectively decrease computational cost as compared with dense connections in DenseNet, and further strengthen the forward feature propagation. DRRNet is evaluated on three object recognition benchmark datasets and an object detection dataset. In comparison with the state-of-the-art, DRRNet achieves a good balance between classification accuracy and computational cost, and optimal detection performance.

Improving license plate recognition via diverse stylistic plate generation

Deep learning-based license plate recognition heavily relies on the quantity and diversity of training images. However, manually collecting diverse license plate images is a highly time-consuming and labor-intensive task. Hence, generating license plate images to supplement training data emerges as a cost-effective solution for recognition enhancement. Previous generation methods treated the style of license plates as discrete features and extracted style features from a referenced image to generate multi-style license plate images. However, these methods are incapable of generating license plates with arbitrary new styles, as the style of the generated images is constrained by the existing images. To solve this problem, we propose to extract style features from randomly sampled vectors in continuous space for diverse stylistic license plate generation. This way, license plates with arbitrary styles can be generated by simply modifying the sampled style vectors. Besides, to provide controllable content information for the generator, we propose an orientation-aware content encoder to extract multi-oriented glyph content features from the input horizontal glyphs. Moreover, a dual-discriminator is proposed to conduct adversarial learning on both color images and image gradients, aiming to mitigate unnatural patterns and ensure realistic generated results. Finally, extensive experiments on multiple public license plate datasets demonstrate that our method can generate realistic and diverse license plates, and the generated images can significantly improve recognition performance by up to 11.5%.

A 4D spontaneous micro‐expression database: Establishment and evaluation

AbstractMicro‐expressions are spontaneous and unconscious facial movements that reveal individuals’ genuine inner emotions. They hold significant potential in various psychological testing fields. As the face is a 3D deformation object, the emergence of facial expression leads to spatial deformation of the face. However, existing databases primarily offer 2D video sequences, limiting descriptions of 3D spatial information related to micro‐expressions. Here, a new micro‐expression database is proposed, which contains 2D image sequences and corresponding 3D point cloud sequences. These samples were classified using both an objective method based on the facial action coding system and a non‐objective emotion classification method that considers video contents and participants’ self‐reports. A variety of feature extraction techniques are applied to 2D data, including traditional algorithms and deep learning methods. Additionally, a novel local curvature‐based algorithm is developed to extract 3D spatio‐temporal deformation features from the 3D data. The authors evaluated the classification accuracies of these two features individually and their fusion results under leave‐one‐subject‐out (LOSO) and tenfold cross‐validation. The results demonstrate that fusing 3D features with 2D features results in improved recognition performance compared to using 2D features alone.

Research Of Singing With Note-name Recognition Algorithm Based On End-to-End Model

With the rapid development of artificial intelligence, and people are not only seeking material needs, but also pursuing happiness and spiritual aspirations. Especially some singing talent shows appear, but there are many controversial results. In order to solve the problem of singing with note-name evaluation and evaluate the singer's singing level in the professional music education of solfeggio and ear training, the paper uses artificial intelligence algorithms to recognize the note-name. Firstly, preprocess the singing sound data, including pre emphasis, framing, and windowing to reduce the impact of noise, and extract the acoustic features of the singing sound, the acoustic speech features are organized into a form suitable for DCNN input and a DCNN model is designed to obtain the pinyin form of the singing sound, the softmax layer adopts an end-to-end CTC structure, which classifies the input singing speech, finds corresponding phoneme sequences, and obtains the output results. The end-to-end CTC structure is used to classify and optimize the recognition process, and finally the state of the obtained features is output. And then singing pronunciation dictionary generates candidate word sequences based on the mapping relationship between phonemes and notes. Finally, based on the acoustic model score, the candidate note sequence with the highest score is obtained through decoder processing, and obtained the singing with note-name result of the singing sound. In order to further improve recognition performance, the paper introduces a new CTC-DCNN acoustic model. In this model, residual blocks can transfer input features to the output part through shortcuts, allowing the multi-layer convolutional speech features to be preserved as much as possible. At the same time, the deep structure can also better achieve the extraction and analysis of speech features. A new and improved CTC-DCNN acoustic model is obtained by optimizing the maxout function. The algorithm proposed in this paper is fair and equal in obtaining information, and no one participates in the entire process of obtaining scores. It is believed that the scoring results obtained in this way should be more objective.

Low-quality image object detection based on reinforcement learning adaptive enhancement

The task of object detection for low-quality images in multiple scenarios remains a challenging task. To alleviate this problem, a novel approach to improve the detection performance of the model on low-quality images using deep reinforcement learning is proposed. The approach differs from methods that involve data labeling and retraining, as the detection results of the model are utilized to formulate a reward function. This enables deep reinforcement learning to transform low-quality images into high-quality images through image enhancement, ultimately leading to an improvement in the model's recognition performance on low-quality images. An image enhancement tool chain (IETC) has been prepared, offering a flexible and lightweight algorithm selection for the Dueling Deep Q-Network-based Tool Selector (DDQN-TS). By adding a “Pass” option, Enable DDQN-TS to learn the relative relationship between performing and not performing image enhancement. To mitigate the potential inconsistency between the output of the detection model and the actual situation caused by negative samples, which can lead to incorrect decisions made by DDQN-TS, the introduction of a “thresh” parameter regulates the level of intervention in target detection by DIE. The experimental results demonstrate the effectiveness and portability of the DIE model in environments with noise, fog, and uneven brightness.

Open-Set Recognition of Individual Cows Based on Spatial Feature Transformation and Metric Learning.

The automated recognition of individual cows is foundational for implementing intelligent farming. Traditional methods of individual cow recognition from an overhead perspective primarily rely on singular back features and perform poorly for cows with diverse orientation distributions and partial body visibility in the frame. This study proposes an open-set method for individual cow recognition based on spatial feature transformation and metric learning to address these issues. Initially, a spatial transformation deep feature extraction module, ResSTN, which incorporates preprocessing techniques, was designed to effectively address the low recognition rate caused by the diverse orientation distribution of individual cows. Subsequently, by constructing an open-set recognition framework that integrates three attention mechanisms, four loss functions, and four distance metric methods and exploring the impact of each component on recognition performance, this study achieves refined and optimized model configurations. Lastly, introducing moderate cropping and random occlusion strategies during the data-loading phase enhances the model's ability to recognize partially visible individuals. The method proposed in this study achieves a recognition accuracy of 94.58% in open-set scenarios for individual cows in overhead images, with an average accuracy improvement of 2.98 percentage points for cows with diverse orientation distributions, and also demonstrates an improved recognition performance for partially visible and randomly occluded individual cows. This validates the effectiveness of the proposed method in open-set recognition, showing significant potential for application in precision cattle farming management.

Enhancing Facial Recognition Accuracy in Low-Light Conditions Using Convolutional Neural Networks

Facial recognition technology has become increasingly everywhere in various domains, from security and surveillance to personal device authentication. However, its effectiveness can be significantly hindered in low-light conditions, where images often lack sufficient illumination for accurate recognition. This study proposes a novel approach to enhance facial recognition accuracy in low-light conditions using Convolutional Neural Networks (CNNs), Deep Retinex Decomposition Network (DRDN), and CenterFace algorithm. The methodology leverages CNNs for robust feature extraction, while DRDN corrects illumination by decomposing images. CenterFace integrates feature fusion and denoising layers for discriminative facial features and noise mitigation. Experimental results demonstrate a remarkable improvement in recognition performance, exceeding 80% accuracy. This approach showcases the potential of CNN-based methods with advanced techniques to enhance reliability in real-world facial recognition applications, particularly in low-light environments.

Research on non-intrusive load identification method based on multi-feature fusion with improved shufflenetv2

Non-intrusive load monitoring is a significant advancement in energy conservation and smart electricity usage as it enables the identification of load status and equipment type without the need for extensive sensing instruments. As one of the important steps of non-intrusive load monitoring, non-intrusive load identification plays a crucial role in the correct identification of electrical appliances. In this paper, a novel approach is presented to amalgamate multiple appliance features into a new identification feature, which is then used for appliance recognition through an improved lightweight deep learning model. This addresses the issue of low accuracy when using a single feature for appliance identification. The core concept involves initially capturing the current data of an appliance during steady state operation and fusing the image features of the appliance encoded by Gramian summation angular field, Gramian difference angular field, and Markov transition field into new recognition features for each appliance using average weighting. Subsequently, a shufflenetv2 lightweight deep learning model based on the squeeze -and-excitation module is used to mine the constructed load feature information for the load classification task. The method is then experimentally validated on the Self-test Dataset, PLAID, and WHITED dataset, resulting in recognition accuracies of 100%, 98.214%, and 99.745%, respectively. These results demonstrate the effectiveness of the proposed method in significantly improving recognition performance compared to the original approach.

A time-delay neural network for ship-radiated noise recognition based on residual block and attention mechanism

Ship-radiated noise recognition remains a challenging issue in underwater acoustic signal processing for a variety of reasons. The ocean ambient noise influences the signal quality, and the scarcity of samples limits the performance of deep learning algorithms. In this study, a self-attention mechanism and residual block-based time-delay neural network (SRTDNN) is proposed to improve the recognition performance of ship-radiated noise. The noise samples are first segmented and transformed into three-dimensional Mel-scale frequency cepstral coefficients (3D-MFCC), which combine static and dynamic characteristics of targets through the concatenation of MFCC, Delta MFCC and Delta-Delta MFCC. Subsequently, 3D-MFCC are fed as input to the SRTDNN. The SRTDNN extracts time-domain features of inputs through the TDNN layers and then assigns different weights to the frame and channel dimensions of the feature maps through two attention mechanisms. The Res Connections and Res2Block are interspersed to facilitate convergence. During training, a time-domain and frequency-domain data augmentation pipeline is designed to alleviate the lack of samples. Experiments with public and real sea experiment datasets are conducted to compare the SRTDNN with acknowledged high-quality recognition methods. The results show that the SRTDNN has a significant improvement in recognition performance. Furthermore, the ablation tests demonstrate the effectiveness of the proposed method.

An oil wear particle identification method based on Wasserstein generative adversarial network and improved CNN using a custom-built optical imaging sensor

The accurate classification of wear particles in lubricating oil holds significant importance for understanding the wear characteristics of lubricated components. However, there are prominent issues, such as blurred particle images, limited identification efficiency, and low generalization capabilities. To solve such issues, this research initially devises a novel online monitoring sensor to collect particle images based on the optical method. Subsequently, an improved Wasserstein Generative Adversarial Network (WGAN) is developed to ameliorate the categorization performance, which is hampered by data imbalance. Lastly, a MobileNetV2-SENet-based Convolutional Neural Network (MSCNN) is proposed to autonomously discover the associations between images and types of wear particles. Experimental results illustrate that our method achieves a recognition accuracy rate of 92.5%. The analysis outcomes between several comparative networks show that our approach can effectively improve recognition performance when limited samples are available. This work offers an effective solution for automatically identifying wear particles in oil and can be a powerful predictive maintenance tool for rotating mechanical equipment.

PCE-Palm: Palm Crease Energy Based Two-Stage Realistic Pseudo-Palmprint Generation

The lack of large-scale data seriously hinders the development of palmprint recognition. Recent approaches address this issue by generating large-scale realistic pseudo palmprints from Bézier curves. However, the significant difference between Bézier curves and real palmprints limits their effectiveness. In this paper, we divide the Bézier-Real difference into creases and texture differences, thus reducing the generation difficulty. We introduce a new palm crease energy (PCE) domain as a bridge from Bézier curves to real palmprints and propose a two-stage generation model. The first stage generates PCE images (realistic creases) from Bézier curves, and the second stage outputs realistic palmprints (realistic texture) with PCE images as input. In addition, we also design a lightweight plug-and-play line feature enhancement block to facilitate domain transfer and improve recognition performance. Extensive experimental results demonstrate that the proposed method surpasses state-of-the-art methods. Under extremely few data settings like 40 IDs (only 2.5% of the total training set), our model achieves a 29% improvement over RPG-Palm and outperforms ArcFace with 100% training set by more than 6% in terms of TAR@FAR=1e-6.

Telugu letters dataset and parallel deep convolutional neural network with a SGD optimizer model for TCR

<p>Because of the rapid growth in technology breakthroughs, including multimedia and cell phones, Telugu character recognition (TCR) has recently become a popular study area. It is still necessary to construct automated and intelligent online TCR models, even if many studies have focused on offline TCR models. The Telugu character dataset construction and validation using an Inception and ResNet-based model are presented. The collection of 645 letters in the dataset includes 18 Achus, 38 Hallus, 35 Othulu, 34×16 Guninthamulu, and 10 Ankelu. The proposed technique aims to efficiently recognize and identify distinctive Telugu characters online. This model's main pre-processing steps to achieve its goals include normalization, smoothing, and interpolation. Improved recognition performance can be attained by using stochastic gradient descent (SGD) to optimize the model's hyperparameters.</p>

RSRNeT: a novel multi-modal network framework for named entity recognition and relation extraction

Named entity recognition (NER) and relation extraction (RE) are two important technologies employed in knowledge extraction for constructing knowledge graphs. Uni-modal NER and RE approaches solely rely on text information for knowledge extraction, leading to various limitations, such as suboptimal performance and low efficiency in recognizing polysemous words. With the development of multi-modal learning, multi-modal named entity recognition (MNER) and multi-modal relation extraction (MRE) have been introduced to improve recognition performance. However, existing MNER and MRE methods often encounter reduced efficiency when the text includes unrelated images. To address this problem, we propose a novel multi-modal network framework for named entity recognition and relation extraction called RSRNeT. In RSRNeT, we focus on extracting visual features more fully and designing a multi-scale visual feature extraction module based on ResNeSt network. On the other hand, we also emphasize fusing multi-modal features more comprehensively while minimizing interference from irrelevant images. To address this issue, we propose a multi-modal feature fusing module based on RoBERTa network. These two modules enable us to learn superior visual and textual representations, reducing errors caused by irrelevant images. Our approach has undergone extensive evaluation and comparison with various baseline models on MNER and MRE tasks. Experimental results show that our method achieves state-of-the-art performance in recall and F1 score on three public datasets: Twitter2015, Twitter2017 and MNRE.

Deep temporal clustering features for speech emotion recognition

Deep clustering is a popular unsupervised technique for feature representation learning. We recently proposed the chunk-based DeepEmoCluster framework for speech emotion recognition (SER) to adopt the concept of deep clustering as a novel semi-supervised learning (SSL) framework, which achieved improved recognition performances over conventional reconstruction-based approaches. However, the vanilla DeepEmoCluster lacks critical sentence-level temporal information that is useful for SER tasks. This study builds upon the DeepEmoCluster framework, creating a powerful SSL approach that leverages temporal information within a sentence. We propose two sentence-level temporal modeling alternatives using either the temporal-net or the triplet loss function, resulting in a novel temporal-enhanced DeepEmoCluster framework to capture essential temporal information. The key contribution to achieving this goal is the proposed sentence-level uniform sampling strategy, which preserves the original temporal order of the data for the clustering process. An extra network module (e.g., gated recurrent unit) is utilized for the temporal-net option to encode temporal information across the data chunks. Alternatively, we can impose additional temporal constraints by using the triplet loss function while training the DeepEmoCluster framework, which does not increase model complexity. Our experimental results based on the MSP-Podcast corpus demonstrate that the proposed temporal-enhanced framework significantly outperforms the vanilla DeepEmoCluster framework and other existing SSL approaches in regression tasks for the emotional attributes arousal, dominance, and valence. The improvements are observed in fully-supervised learning or SSL implementations. Further analyses validate the effectiveness of the proposed temporal modeling, showing (1) high temporal consistency in the cluster assignment, and (2) well-separated emotional patterns in the generated clusters.

Long Dialogue Emotion Detection Based on Commonsense Knowledge Graph Guidance

Dialogue emotion detection is always challenging due to human subjectivity and the randomness of dialogue content. In a conversation, the emotion of each person often develops via a cumulative process, which can be influenced by many elements of uncertainty. Much commonsense knowledge influences people's emotions imperceptibly, such as experiential or habitual knowledge. In the process of conversation, this commonsense knowledge information can be used to enrich the semantic information of each utterance and improve the accuracy of emotion recognition. In this paper, we propose a growing graph model for dialogues emotion detection based on retrieval of external knowledge atlas ATOMIC from local and global respectively, which can effectively represent the dialogues as a process variable in a sequence and the correlation among utterances also can be represented by the graph model. In particular, 1) we introduce a common sense knowledge graph for linking the commonsense knowledge retrieved from external knowledge atlas ATOMIC, which can effectively add auxiliary information to improve the performance of each utterance's representation. 2) We propose a novel self-supervised learning method for extracting the latent topic of each dialogue. Based on this design, we also propose an effective optimization mechanism to make the representation (embedding) of latent topic has a better distinction for the next operation. 3) Finally, the cross-attention module is utilized to combine the utterances' features and the latent conversation topic information. The attention mechanism can effectively use topic information to supplement the representation of utterances and improve recognition performance. The model is tested on three popular datasets in dialogue emotion detection and is empirically demonstrated to outperform the state-of-the-art approaches. Meanwhile, to demonstrate the performance of our approach, we also build a long dialogue dataset. The average length of each conversation is over 50 utterances. The final experimental results also demonstrate the superior performance of our approach.

Micro-expression recognition based on multi-scale 3D residual convolutional neural network

<abstract><p>In demanding application scenarios such as clinical psychotherapy and criminal interrogation, the accurate recognition of micro-expressions is of utmost importance but poses significant challenges. One of the main difficulties lies in effectively capturing weak and fleeting facial features and improving recognition performance. To address this fundamental issue, this paper proposed a novel architecture based on a multi-scale 3D residual convolutional neural network. The algorithm leveraged a deep 3D-ResNet50 as the skeleton model and utilized the micro-expression optical flow feature map as the input for the network model. Drawing upon the complex spatial and temporal features inherent in micro-expressions, the network incorporated multi-scale convolutional modules of varying sizes to integrate both global and local information. Furthermore, an attention mechanism feature fusion module was introduced to enhance the model's contextual awareness. Finally, to optimize the model's prediction of the optimal solution, a discriminative network structure with multiple output channels was constructed. The algorithm's performance was evaluated using the public datasets SMIC, SAMM, and CASME Ⅱ. The experimental results demonstrated that the proposed algorithm achieves recognition accuracies of 74.6, 84.77 and 91.35% on these datasets, respectively. This substantial improvement in efficiency compared to existing mainstream methods for extracting micro-expression subtle features effectively enhanced micro-expression recognition performance and increased the accuracy of high-precision micro-expression recognition. Consequently, this paper served as an important reference for researchers working on high-precision micro-expression recognition.</p></abstract>

Multiscale Human Activity Recognition and Anticipation Network.

Deep convolutional neural networks have been leveraged to achieve huge improvements in video understanding and human activity recognition performance in the past decade. However, most existing methods focus on activities that have similar time scales, leaving the task of action recognition on multiscale human behaviors less explored. In this study, a two-stream multiscale human activity recognition and anticipation (MS-HARA) network is proposed, which is jointly optimized using a multitask learning method. The MS-HARA network fuses the two streams of the network using an efficient temporal-channel attention (TCA)-based fusion approach to improve the model's representational ability for both temporal and spatial features. We investigate the multiscale human activities from two basic categories, namely, midterm activities and long-term activities. The network is designed to function as part of a real-time processing framework to support interaction and mutual understanding between humans and intelligent machines. It achieves state-of-the-art results on several datasets for different tasks and different application domains. The midterm and long-term action recognition and anticipation performance, as well as the network fusion, are extensively tested to show the efficiency of the proposed network. The results show that the MS-HARA network can easily be extended to different application domains.

Fusion recognition of palmprint and palm vein based on modal correlation.

Biometric authentication prevents losses from identity misuse in the artificial intelligence (AI) era. The fusion method integrates palmprint and palm vein features, leveraging their stability and security and enhances counterfeiting prevention and overall system efficiency through multimodal correlations. However, most of the existing multi-modal palmprint and palm vein feature extraction methods extract only feature information independently from different modalities, ignoring the importance of the correlation between different modal samples in the class to the improvement of recognition performance. In this study, we addressed the aforementioned issues by proposing a feature-level joint learning fusion approach for palmprint and palm vein recognition based on modal correlations. The method employs a sparse unsupervised projection algorithm with a "purification matrix" constraint to enhance consistency in intra-modal features. This minimizes data reconstruction errors, eliminating noise and extracting compact, and discriminative representations. Subsequently, the partial least squares algorithm extracts high grayscale variance and category correlation subspaces from each modality. A weighted sum is then utilized to dynamically optimize the contribution of each modality for effective classification recognition. Experimental evaluations conducted for five multimodal databases, composed of six unimodal databases including the Chinese Academy of Sciences multispectral palmprint and palm vein databases, yielded equal error rates (EER) of 0.0173%, 0.0192%, 0.0059%, 0.0010%, and 0.0008%. Compared to some classical methods for palmprint and palm vein fusion recognition, the algorithm significantly improves recognition performance. The algorithm is suitable for identity recognition in scenarios with high security requirements and holds practical value.