Improve Recognition Performance Research Articles

Emotion recognition and mood along the menstrual cycle

Previous studies have demonstrated menstrual cycle dependent changes in the recognition of facial emotional expressions, specifically the expression of fear, anger, sadness or disgust. While some studies demonstrate an improvement of emotion recognition performance during the peri-ovulatory phase, when estradiol levels peak, other studies demonstrate a deterioration of emotion recognition performance during the mid-luteal phase, when progesterone levels peak. It has been hypothesized, that these changes in emotion recognition performance mirror mood changes along the menstrual cycle. In the present study, we investigate, whether changes in emotion recognition performance along the menstrual cycle are mediated by mood changes along the menstrual cycle. In a combined cross-sectional and longitudinal study design, two large samples of women completed an emotion recognition task, as well as several mood questionnaires during their menses, peri-ovulatory or mid-luteal cycle phase. 65 women completed the task thrice, once during each cycle phase, order counterbalanced. In order to control for potential learning effects, a sample of 153 women completed the task only once in one of the three cycle phases. In both samples, results demonstrated no significant changes in emotion recognition performance along the menstrual cycle, irrespective of the performance measure investigated (accuracy, reaction time, frequency of emotion classifications) and irrespective of the emotion displayed. Bayesian statistics provided very strong evidence for the null hypothesis, that emotion recognition does not change along the menstrual cycle. There was also no moderation of emotion recognition changes along the menstrual cycle by mood changes along the menstrual cycle. Mood changes along the menstrual cycle followed the expected pattern with highest positive affect and least premenstrual symptoms around ovulation and lowest positive affect, but strongest premenstrual symptoms during menses. Interestingly, premenstrual symptoms were negatively related to estradiol, suggesting a protective effect of estrogen during the luteal cycle phase against mood worsening during the premenstrual phase.

Few-shot person re-identification based on Feature Set Augmentation and Metric Fusion

Person re-identification identifies pedestrians by analyzing image information from surveillance videos. However, it faces challenges like occlusion, changing lighting, and costly annotation. Thus, it is often performed in a few-shot environment with limited images. In response to the problem of insufficient available pedestrian images in person re-identification, a few-shot person re-identification method based on feature set augmentation and metric fusion is proposed. In this work, firstly, a feature augmentation method is introduced into the feature embedding module. This method introduces multi-head self-attention in different feature extraction layers and spatial attention in feature fusion of different feature extraction layers, which can extract more diverse and discriminative pedestrian features. Secondly, a dual metric method combining Euclidean and cosine distance is proposed in the metric module to comprehensively measure the absolute spatial distance and directional difference of pedestrian features. In this way, the reliability of pedestrian similarity measurement is improved. Then, pedestrian feature similarity scores are obtained separately using the dual metric and relation metric methods. Finally, the combined metric score is obtained by weighted fusion, and the combined metric score is used to construct the joint loss to realize the overall optimization and training of the network. Experimental results on three small datasets, Market-mini, Duke-mini, and MSMT17-mini, show that the proposed method significantly improves recognition performance compared to other few-shot learning algorithms. Specifically, in scenarios 5-way 1-shot and 5-way 5-shot, the average recognition accuracies are 92.54% and 96.99%, 87.93% and 96.08%, and 71.68% and 84.51%, respectively.

Mean Teacher-Based Cross-Domain Activity Recognition Using WiFi Signals

WiFi Channel State Information (CSI)-based activity recognition has initiated a great many studies because of wide availability and privacy protection. However, general recognition approaches still struggle to generalize beyond the source domain of training data, i.e., well-trained models might not be suitable to target data with unseen subjects or environments. Existing solutions, such as few-shot learning-based and data augmentation-based approaches, either require a few labeled target samples which is difficult to be collected especially for old target users, or inappropriately treat augmented samples with different amounts of noise. To overcome these limitations, we propose a Mean Teacher-based cross-domain human activity recognition framework using WiFi CSI, WiTeacher. In this framework, to address shift between source and target domains, we built a label smoothing-based classification loss, where the input data are the target-like samples generated by StyleGAN, and corresponding label values are dynamically adjusted by our designed adaptive label smoothing method. To enhance the model robustness, we devise a sample relation-based consistency regularization term to keep the distances of the two samples with and without perturbations invariant, which can exploit the relationships between samples to improve recognition performance. The experiments illustrate that WiTeacher achieves obvious gains without requiring any annotation data from the target domain.

Space-and-speaker-aware acoustic modeling with effective data augmentation for recognition of multi-array conversational speech

We propose a space-and-speaker-aware (SSA) approach to acoustic modeling (AM), denoted as SSA-AM, to improve system performances of automatic speech recognition (ASR) in distant multi-array conversational scenarios. In contrast to conventional AM which only uses spectral features from a target speaker as inputs, the inputs to SSA-AM consists of speech features from both the target and interfering speakers, which contain discriminative information from different speakers, including spatial information embedded in interaural phase differences (IPDs) between individual interfering speakers and the target speaker. In the proposed SSA-AM framework, we explore four acoustic model architectures consisting of different combinations of four neural networks, namely deep residual network, factorized time delay neural network, self-attention and residual bidirectional long short-term memory neural network. Various data augmentation techniques are adopted to expand the training data to include different options of beamformed speech obtained from multi-channel speech enhancement. Evaluated on the recent CHiME-6 Challenge Track 1, our proposed SSA-AM framework achieves consistent recognition performance improvements when compared with the official baseline acoustic models. Furthermore, SSA-AM outperforms acoustic models without explicitly using the space and speaker information. Finally, our data augmentation schemes are shown to be especially effective for compact model designs. Code is released at https://github.com/coalboss/SSA_AM.

Contrastive self-supervised representation learning without negative samples for multimodal human action recognition.

Action recognition is an important component of human-computer interaction, and multimodal feature representation and learning methods can be used to improve recognition performance due to the interrelation and complementarity between different modalities. However, due to the lack of large-scale labeled samples, the performance of existing ConvNets-based methods are severely constrained. In this paper, a novel and effective multi-modal feature representation and contrastive self-supervised learning framework is proposed to improve the action recognition performance of models and the generalization ability of application scenarios. The proposed recognition framework employs weight sharing between two branches and does not require negative samples, which could effectively learn useful feature representations by using multimodal unlabeled data, e.g., skeleton sequence and inertial measurement unit signal (IMU). The extensive experiments are conducted on two benchmarks: UTD-MHAD and MMAct, and the results show that our proposed recognition framework outperforms both unimodal and multimodal baselines in action retrieval, semi-supervised learning, and zero-shot learning scenarios.

A Customized ECA-CRNN Model for Emotion Recognition Based on EEG Signals

Electroencephalogram (EEG) signals are electrical signals generated by changes in brain potential. As a significant physiological signal, EEG signals have been applied in various fields, including emotion recognition. However, current deep learning methods based on EEG signals for emotion recognition lack consideration of important aspects and comprehensive analysis of feature extraction interactions. In this paper, we propose a novel model named ECA-CRNN for emotion recognition using EEG signals. Our model integrates the efficient channel attention (ECA-Net) module into our modified combination of a customized convolutional neural network (CNN) and gated circulation unit (GRU), which enables more comprehensive feature extraction, enhances the internal relationship between frequency bands and improves recognition performance. Additionally, we utilize four-dimensional data as input to our model, comprising temporal, spatial and frequency information. The test on the DEAP dataset demonstrates that it enhances the recognition accuracy of EEG signals in both arousal and valence to 95.70% and 95.33%, respectively, while also reducing the standard deviation during five-fold cross-validation to 1.16 and 1.45 for arousal and valence, respectively, surpassing most methods.

MLST-Former: Multi-Level Spatial-Temporal Transformer for Group Activity Recognition

Group activity recognition, which aims to simultaneously understand individual action and group activity in video clips, plays a fundamental role in computer vision and video analysis. In this paper, we propose a novel relational inference framework, termed MLST-Former, for individual action and group activity recognition, capturing the various degrees of spatial-temporal dynamic interactions adaptively and jointly among actors and generating reasonable group representations. Specifically, we first design a multi-level spatial-temporal Transformer to capture the miscellaneous actors’ spatial-temporal contextual information to deal with unbalanced interactions between actors. Furthermore, our proposed network is capable of fully mining long-range spatial-temporal dependencies with the virtue of the merge function and cross attention mechanism. We then propose an inter-frame gating fusion mechanism (IGFM) to selectively aggregate the temporal and structural features of the interacting actors. A new multi-task learning strategy, consisting of the classification cost of individual actions and group activities, is also developed. Moreover, we adopt the motion trajectory branch to provide complementary dynamic features for improving recognition performance. A series of ablation studies demonstrate the effectiveness and respective contributions of the different components within the proposed method. Extensive experiments on four public GAR datasets clearly show that our approach can achieve very competitive performance by comparing them with state-of-the-art methods.

Semantically preserving adversarial unsupervised domain adaptation network for improving disease recognition from chest x-rays.

Supervised deep learning methods have been successfully applied in medical imaging. However, training deep learning systems often requires ample annotated data. Due to cost and time restrictions, not all collected medical images, e.g., chest x-rays (CXRs), can be labeled in practice. To classify these unlabeled images, a solution may involve adopting a model trained with sufficient labeled data in relevant domains (with both source and target being CXRs). However, domain shift may cause the trained model not able to generalize well on unlabeled target datasets. This work aims to develop a novel unsupervised domain adaptation (UDA) framework to improve recognition performance on unlabeled target data. We present a semantically preserving adversarial UDA network, i.e., SPA-UDA net, with the potential to bridge the domain gap, by reconstructing the images in the target domain via an adversarial encode-and-reconstruct translation architecture. To preserve the class-specific semantic information (i.e., with or without disease) of the original images when translating, a semantically consistent framework is embedded. This framework is designed to guarantee that fine-grained disease-related information on the original images can be safely transferred. Furthermore, the proposed SPA-UDA net does not require paired images from source and target domains when training, which reduces the cost of arranging data significantly and is ideal for UDA. We evaluate the proposed SPA-UDA net on two public CXR datasets for lung disease recognition. The experimental results show that the proposed framework achieves significant performance improvements compared to other state-of-the-art UDA methods.

Named Entity Identification in the Power Dispatch Domain Based on RoBERTa-Attention-FL Model

Named entity identification is an important step in building a knowledge graph of the grid domain, which contains a certain number of nested entities. To address the issue of nested entities in the Chinese power dispatching domain’s named entity recognition, we propose a RoBERTa-Attention-FL model. This model effectively recognizes nested entities using the span representation annotation method. We extract the output values from RoBERTa’s middle 4–10 layers, obtain syntactic information from the Transformer Encoder layers via the multi-head self-attention mechanism, and integrate it with deep semantic information output from RoBERTa’s last layer. During training, we use Focal Loss to mitigate the sample imbalance problem. To evaluate the model’s performance, we construct named entity recognition datasets for flat and nested entities in the power dispatching domain annotated with actual power operation data, and conduct experiments. The results indicate that compared to the baseline model, the RoBERTa-Attention-FL model significantly improves recognition performance, increasing the F1-score by 4.28% to 90.35%, with an accuracy rate of 92.53% and a recall rate of 88.12%.

Discover human poses similarity and action recognition based on machine learning

In the computer vision field, human action recognition depending on pose estimation recently made considerable progress, especially by using deep learning, which improves recognition performance. Therefore, it has been employed in various applications, including sports and physical activity follow-up. This paper presents a technique for recognizing the human posture in different images and matching their pose similarity. This aims to evaluate the viability of employing computer vision techniques to verify a person's body pose during exercise and determine whether the pose is executed properly. Exercise is one strategy we use to maintain our health throughout life. Gymnastics and yoga are two examples of this type of exercise. The proposed algorithm identifies human action by recognizing the body's key points. The OpenPose library has been used to detect 18 key points of the human body. The action classification task is performed using the support vector machine (SVM) algorithm. Then, the algorithm computes the similarity of the human pose by comparing a model image to a test image to determine the matching score. Evaluations show that our method can perform at a competitive or state-of-the-art performance on a number of body pose datasets.

Multimodal Biometric System Based on the Fusion in Score of Fingerprint and Online Handwritten Signature

Abstract Multimodal biometrics is the technique of using multiple modalities on a single system. This allows us to overcome the limitations of unimodal systems, such as the inability to acquire data from certain individuals or intentional fraud, while improving recognition performance. In this paper, a study of score normalization and its impact on the performance of the system is performed. The fusion of scores requires prior normalisation before applying a weighted sum fusion that separates impostor and genuine scores into a common interval with close ranges. The experiments were carried out on three biometric databases. The results show that the proposed strategy performs very encouragingly, especially in combination with Empirical Modal Decomposition (EMD). The proposed fusion system shows good performance. The best result is obtained by merging the globality online signature and fingerprint where an EER of 1.69 % is obtained by normalizing the scores according to the Min-Max method.

Leveraging Hand-Object Interactions in Assistive Egocentric Vision.

Egocentric vision holds great promise for increasing access to visual information and improving the quality of life for blind people. While we strive to improve recognition performance, it remains difficult to identify which object is of interest to the user; the object may not even be included in the frame due to challenges in camera aiming without visual feedback. Also, gaze information, commonly used to infer the area of interest in egocentric vision, is often not dependable. However, blind users tend to include their hand either interacting with the object they wish to recognize or simply placing it in proximity for better camera aiming. We propose a method that leverages the hand as the contextual information for recognizing an object of interest. In our method, the output of a pre-trained hand segmentation model is infused to later convolutional layers of our object recognition network with separate output layers for localization and classification. Using egocentric datasets from sighted and blind individuals, we show that the hand-priming achieves more accurate localization than other approaches that encode hand information. Given only object centers along with labels, our method achieves comparable classification performance to the state-of-the-art method that uses bounding boxes with labels.

A novel Jinnan individual cattle recognition approach based on mutual attention learning scheme

The accurate recognition of individual Jinnan cattle is crucial for efficient breeding. However, existing ear tagging and machine learning approaches are inefficient, and deep learning models with a significant number of parameters can be computationally expensive. To address these challenges, this study proposes an innovative mutual-attention learning method for efficiently recognizing individual Jinnan cattle. This method employs two attention-enhanced student networks that collaborate and exchange knowledge on cross-dimensional attention during training, achieving improved recognition performance compared to counterparts without mutual learning. Additionally, smaller student networks can achieve improved recognition performance with much less computational cost, leveraging knowledge from larger networks. Furthermore, the proposed approach integrates attention modules such as block attention module (BAM), convolutional BAM (CBAM), and triplet attention units to capture spatial and channel attention for obtaining robust feature representations, enhancing performance. Our proposed model achieves state-of-the-art classification accuracy of 98.91%, surpassing other models in both accuracy and efficiency. Experiments also demonstrate its superiority by improving accuracy by over 2.97% and reducing loss value by over 50% compared to existing methods. Overall, the proposed approach has great potential to revolutionize the recognition of individual Jinnan cattle.

TSML: A New Pig Behavior Recognition Method Based on Two-Stream Mutual Learning Network.

Changes in pig behavior are crucial information in the livestock breeding process, and automatic pig behavior recognition is a vital method for improving pig welfare. However, most methods for pig behavior recognition rely on human observation and deep learning. Human observation is often time-consuming and labor-intensive, while deep learning models with a large number of parameters can result in slow training times and low efficiency. To address these issues, this paper proposes a novel deep mutual learning enhanced two-stream pig behavior recognition approach. The proposed model consists of two mutual learning networks, which include the red-green-blue color model (RGB) and flow streams. Additionally, each branch contains two student networks that learn collaboratively to effectively achieve robust and rich appearance or motion features, ultimately leading to improved recognition performance of pig behaviors. Finally, the results of RGB and flow branches are weighted and fused to further improve the performance of pig behavior recognition. Experimental results demonstrate the effectiveness of the proposed model, which achieves state-of-the-art recognition performance with an accuracy of 96.52%, surpassing other models by 2.71%.

Recognition of Ellipsoid-like Herbaceous Tibetan Medicinal Materials Using DenseNet with Attention and ILBP-Encoded Gabor Features.

Tibetan medicinal materials play a significant role in Tibetan culture. However, some types of Tibetan medicinal materials share similar shapes and colors, but possess different medicinal properties and functions. The incorrect use of such medicinal materials may lead to poisoning, delayed treatment, and potentially severe consequences for patients. Historically, the identification of ellipsoid-like herbaceous Tibetan medicinal materials has relied on manual identification methods, including observation, touching, tasting, and nasal smell, which heavily rely on the technicians' accumulated experience and are prone to errors. In this paper, we propose an image-recognition method for ellipsoid-like herbaceous Tibetan medicinal materials that combines texture feature extraction and a deep-learning network. We created an image dataset consisting of 3200 images of 18 types of ellipsoid-like Tibetan medicinal materials. Due to the complex background and high similarity in the shape and color of the ellipsoid-like herbaceous Tibetan medicinal materials in the images, we conducted a multi-feature fusion experiment on the shape, color, and texture features of these materials. To leverage the importance of texture features, we utilized an improved LBP (local binary pattern) algorithm to encode the texture features extracted by the Gabor algorithm. We inputted the final features into the DenseNet network to recognize the images of the ellipsoid-like herbaceous Tibetan medicinal materials. Our approach focuses on extracting important texture information while ignoring irrelevant information such as background clutter to eliminate interference and improve recognition performance. The experimental results show that our proposed method achieved a recognition accuracy of 93.67% on the original dataset and 95.11% on the augmented dataset. In conclusion, our proposed method could aid in the identification and authentication of ellipsoid-like herbaceous Tibetan medicinal materials, reducing errors and ensuring the safe use of Tibetan medicinal materials in healthcare.

Joint Modal Alignment and Feature Enhancement for Visible-Infrared Person Re-Identification.

Visible-infrared person re-identification aims to solve the matching problem between cross-camera and cross-modal person images. Existing methods strive to perform better cross-modal alignment, but often neglect the critical importance of feature enhancement for achieving better performance. Therefore, we proposed an effective method that combines both modal alignment and feature enhancement. Specifically, we introduced Visible-Infrared Modal Data Augmentation (VIMDA) for visible images to improve modal alignment. Margin MMD-ID Loss was also used to further enhance modal alignment and optimize model convergence. Then, we proposed Multi-Grain Feature Extraction (MGFE) Structure for feature enhancement to further improve recognition performance. Extensive experiments have been carried out on SYSY-MM01 and RegDB. The result indicates that our method outperforms the current state-of-the-art method for visible-infrared person re-identification. Ablation experiments verified the effectiveness of the proposed method.

Improved YOLOv5 Network for Real-Time Object Detection in Vehicle-Mounted Camera Capture Scenarios.

Object detection in the process of driving is a convenient and efficient task. However, due to the complex transformation of the road environment and vehicle speed, the scale of the target will not only change significantly but also be accompanied by the phenomenon of motion blur, which will have a significant impact on the detection accuracy. In practical application scenarios, it is difficult for traditional methods to simultaneously take into account the need for real-time detection and high accuracy. To address the above problems, this study proposes an improved network based on YOLOv5, taking traffic signs and road cracks as detection objects and conducting separate research. This paper proposes a GS-FPN structure to replace the original feature fusion structure for road cracks. This structure integrates the convolutional block attention model (CBAM) based on bidirectional feature pyramid networks (Bi-FPN) and introduces a new lightweight convolution module (GSConv) to reduce the information loss of the feature map, enhance the expressive ability of the network, and ultimately achieve improved recognition performance. For traffic signs, a four-scale feature detection structure is used to increase the detection scale of shallow layers and improve the recognition accuracy for small targets. In addition, this study has combined various data augmentation methods to improve the robustness of the network. Through experiments using 2164 road crack datasets and 8146 traffic sign datasets made by LabelImg, compared to the baseline model (YOLOv5s), the modified YOLOv5 network improves the mean average precision (mAP) result of the road crack dataset and small targets in the traffic sign dataset by 3% and 12.2%, respectively.

Robust speech recognition based on deep learning for sports game review

Abstract To verify the feasibility of robust speech recognition based on deep learning in sports game review. In this paper, a robust speech recognition model is built based on the generative adversarial network GAN algorithm according to the deep learning model. And the loss function, optimization function and noise reduction front-end are introduced in the model to achieve the optimization of speech extraction features through denoising process to ensure that accurate speech review data can be derived even in the game scene under noisy environment. Finally, the experiments are conducted to verify the four directions of the model algorithm by comparing the speech features MFCC, FBANK and WAVE. The experimental results show that the speech recognition model trained by the GSDNet model algorithm can reach 89% accuracy, 56.24% reduction of auxiliary speech recognition word error rate, 92.61% accuracy of speech feature extraction, about 62.19% reduction of training sample data volume, and 94.75% improvement of speech recognition performance in the speech recognition task under noisy environment. It shows that the robust speech recognition based on deep learning can be applied to sports game reviews, and also can provide accurate voice review information from the noisy sports game scene, and also broaden the application area for deep learning models.

Cross-subject EEG emotion recognition using multi-source domain manifold feature selection

Recent researches on emotion recognition suggests that domain adaptation, a form of transfer learning, has the capability to solve the cross-subject problem in Affective brain-computer interface (aBCI) field. However, traditional domain adaptation methods perform single to single domain transfer or simply merge different source domains into a larger domain to realize the transfer of knowledge, resulting in negative transfer. In this study, a multi-source transfer learning framework was proposed to promote the performance of multi-source electroencephalogram (EEG) emotion recognition. The method first used the data distribution similarity ranking (DDSA) method to select the appropriate source domain for each target domain off-line, and reduced data drift between domains through manifold feature mapping on Grassmann manifold. Meanwhile, the minimum redundancy maximum correlation algorithm (mRMR) was employed to select more representative manifold features and minimized the conditional distribution and marginal distribution of the manifold features, and then learned the domain-invariant classifier by summarizing structural risk minimization (SRM). Finally, the weighted fusion criterion was applied to further improve recognition performance. We compared our method with several state-of-the-art domain adaptation techniques using the SEED and DEAP dataset. Results showed that, compared with the conventional MEDA algorithm, the recognition accuracy of our proposed algorithm on SEED and DEAP dataset were improved by 6.74% and 5.34%, respectively. Besides, compared with TCA, JDA, and other state-of-the-art algorithms, the performance of our proposed method was also improved with the best average accuracy of 86.59% on SEED and 64.40% on DEAP. Our results demonstrated that the proposed multi-source transfer learning framework is more effective and feasible than other state-of-the-art methods in recognizing different emotions by solving the cross-subject problem.

Identification of depression state based on multi‐scale acoustic features in interrogation environment

AbstractDepression diagnosis based on speech signals has the advantages of non‐invasiveness, low cost, and few restrictions on portability. The research on the recognition of the depression state is carried out based on the acoustic information in the speech signal. Aiming at the interview dialogue speech in the consultation environment, a hierarchical attention temporal convolutional network (HATCN) acoustic depression recognition model is proposed. For sentence acoustic feature learning, a regional attention mechanism is introduced to extract multi‐scale sentence features; for segment acoustic feature extraction, the traditional attention mechanism is used to calculate, which is in line with human cognitive mechanism. In addition, a periodic focal loss function is introduced to address the imbalance of positive and negative samples in depression diagnosis. Experiments show that the proposed acoustic depression recognition model has a certain improvement in recognition performance compared with other methods. At the same time, the influence of noise on the recognition of acoustic depression in the real consultation environment is analysed through experiments, and the data enhancement is carried out utilising speech noise, which proves the effectiveness of the data expansion of speech noise.