Recognition Datasets Research Articles

Named entity recognition in aerospace based on multi-feature fusion transformer

In recent years, along with the rapid development in the domain of artificial intelligence and aerospace, aerospace combined with artificial intelligence is the future trend. As an important basic tool for Natural Language Processing, Named Entity Recognition technology can help obtain key relevant knowledge from a large number of aerospace data. In this paper, we produced an aerospace domain entity recognition dataset containing 30 k sentences in Chinese and developed a named entity recognition model that is Multi-Feature Fusion Transformer (MFT), which combines features such as words and radicals to enhance the semantic information of the sentences. In our model, the double Feed-forward Neural Network is exploited as well to ensure MFT better performance. We use our aerospace dataset to train MFT. The experimental results show that MFT has great entity recognition performance, and the F1 score on aerospace dataset is 86.10%.

An investigation into the reliability of speaker recognition schemes: analysing the impact of environmental factors utilising deep learning techniques

This paper studies the performance and reliability of deep learning-based speaker recognition schemes under various recording situations and background noise presence. The study uses the Speaker Recognition Dataset offered in the Kaggle website, involving audio recordings from different speakers, and four scenarios with various combinations of speakers. In the first scenario, the scheme achieves discriminating capability and high accuracy in identifying speakers without taking into account outside noise, having roughly one area under the ROC curve. Nevertheless, in the second scenario, with background noise added to the recording, accuracy decreases, and misclassifications increase. However, the scheme still reveals good discriminating power, with ROC areas ranging from 0.77 to 1.

GLOCAL: A self-supervised learning framework for global and local motion estimation

Motions in videos are typically a mixture of local dynamic object motions and global camera motion, which are inconsistent in some cases, and even interfere with each other, causing difficulties in various downstream applications, such as video stabilization that requires the global motion, and action recognition that consumes local motions. Therefore, it is crucial to estimate them separately. Existing methods separate two motions from the mixed motion fields, such as optical flow. However, the quality of mixed motion determines the higher bounds of the performance. In this work, we propose a framework, GLOCAL, to directly estimate global and local motions simultaneously from adjacent frames in a self-supervised manner. Our GLOCAL consists of a Global Motion Estimation (GME) module and a Local Motion Estimation (LME) module. The GME module involves a mixed motion estimation backbone, an implicit bottleneck structure for feature dimension reduction, and an explicit bottleneck for global motion recovery based on the global motion bases with foreground mask under the training guidance of proposed global reconstruction loss. An attention U-Net is adopted for LME which produces local motions while excluding motion of irrelevant regions under the guidance of proposed local reconstruction loss. Our method can achieve better performance than the existing methods on the homography estimation dataset DHE and the action recognition dataset NCAA and UCF-101.

Kernel Probabilistic Dependent-Independent Canonical Correlation Analysis

There is growing interest in developing linear/nonlinear feature fusion methods that fuse the elicited features from two different sources of information for achieving a higher recognition rate. In this regard, canonical correlation analysis (CCA), cross-modal factor analysis, and probabilistic CCA (PCCA) have been introduced to better deal with data variability and uncertainty. In our previous research, we formerly developed the kernel version of PCCA (KPCCA) to capture both nonlinear and probabilistic relation between the features of two different source signals. However, KPCCA is only able to estimate latent variables, which are statistically correlated between the features of two independent modalities. To overcome this drawback, we propose a kernel version of the probabilistic dependent-independent CCA (PDICCA) method to capture the nonlinear relation between both dependent and independent latent variables. We have compared the proposed method to PDICCA, CCA, KCCA, cross-modal factor analysis (CFA), and kernel CFA methods over the eNTERFACE and RML datasets for audio-visual emotion recognition and the M2VTS dataset for audio-visual speech recognition. Empirical results on the three datasets indicate the superiority of both the PDICCA and Kernel PDICCA methods to their counterparts.

A novel parameter dense three-dimensional convolution residual network method and its application in classroom teaching.

Improving the rationality and accuracy of classroom quality analysis is crucial in modern education. Traditional methods, such as questionnaires and manual recordings, are resource-intensive and subjective, leading to inconsistent results. As a solution, computer vision (CV) technologies have emerged as powerful tools for real-time classroom monitoring. This study proposes a novel Dense 3D Convolutional Residual Network (D3DCNN_ResNet) to recognize students' expressions and behaviors in English classrooms. The proposed method combines Single Shot Multibox Detector (SSD) for target detection with an improved D3DCNN_ResNet model. The network applies 3D convolution in both spatial and temporal domains, with shortcut connections from residual blocks to increase network depth. Dense connections are introduced to enhance the flow of high- and low-level features. The model was tested on two datasets: the CK+ dataset for expression recognition and the KTH dataset for behavior recognition. The experiments show that the proposed method is highly efficient in optimizing model training and improving recognition accuracy. On the CK+ dataset, the model achieved an expression recognition accuracy of 97.94%, while on the KTH dataset, the behavior recognition accuracy reached 98.86%. The combination of residual blocks and dense connections reduced feature redundancy and improved gradient flow, leading to better model performance. The results demonstrate that the D3DCNN_ResNet is well-suited for classroom quality analysis and has the potential to enhance teaching strategies by providing real-time feedback on student engagement.

E‐Speech: Development of a Dataset for Speech Emotion Recognition and Analysis

Speech emotion recognition plays a crucial role in analyzing psychological disorders, behavioral decision‐making, and human‐machine interaction applications. However, the majority of current methods for speech emotion recognition heavily rely on data‐driven approaches, and the scarcity of emotion speech datasets limits the progress in research and development of emotion analysis and recognition. To address this issue, this study introduces a new English speech dataset specifically designed for emotion analysis and recognition. This dataset consists of 5503 voices from over 60 English speakers in different emotional states. Furthermore, to enhance emotion analysis and recognition, fast Fourier transform (FFT), short‐time Fourier transform (STFT), mel‐frequency cepstral coefficients (MFCCs), and continuous wavelet transform (CWT) are employed for feature extraction from the speech data. Utilizing these algorithms, the spectrum images of the speeches are obtained, forming four datasets consisting of different speech feature images. Furthermore, to evaluate the dataset, 16 classification models and 19 detection algorithms are selected. The experimental results demonstrate that the majority of classification and detection models achieve exceptionally high recognition accuracy on this dataset, confirming its effectiveness and utility. The dataset proves to be valuable in advancing research and development in the field of emotion recognition.

A Neuro Phenotypic Evolution Algorithm for Recognizing Human Motion Type

Living in the modern world requires having a precise, intelligent system that may be suggested to do various activities. Due to the scalability of AI algorithms, we have proposed a phenotypic Evolutionary Algorithm (EA)-based system to assist the Artificial Neural Network algorithm (ANN) in the learning process. Combining the two strategies can result in a smart neuro-evolutionary model that is effective in accomplishing significant duties in various domains. The suggested multi-layer neural algorithm's design creates the conditions for learning via the EA’s processes of crossover, mutation, and selection. To aid in the selection and crossover processes, the learning process phase breaks up the original ANN into multiple ANNs according to the number of hidden layers. ANNs are ranked from worst to best in the selection phase based on the soring function that is applied to the fitness list. The fitness list retains each ANN’s accuracy even after breaking apart the original ANN. The crossover procedure is then applied between the two worst and best ANNs. Mutation provides a means of improvement for the less effective ANNs. Following completion of these processes, the ANN algorithms are combined to create a single ANN algorithm. The Vicon mobile robot (SCITOS G5) system’s multi-dimensional data, which extracted both aggressive and typical human movement, as well as Human Activity Recognition (HAR) datasets extracted by smartphones, both have been applied using the suggested method. The system achieved a high performance and efficiency rate on the intended recognition problem

A Hybrid Learning-Architecture for Mental Disorder Detection Using Emotion Recognition.

Mental illness has grown to become a prevalent and global health concern that affects individuals across various demographics. Timely detection and accurate diagnosis of mental disorders are crucial for effective treatment and support as late diagnosis could result in suicidal, harmful behaviors and ultimately death. To this end, the present study introduces a novel pipeline for the analysis of facial expressions, leveraging both the AffectNet and 2013 Facial Emotion Recognition (FER) datasets. Consequently, this research goes beyond traditional diagnostic methods by contributing a system capable of generating a comprehensive mental disorder dataset and concurrently predicting mental disorders based on facial emotional cues. Particularly, we introduce a hybrid architecture for mental disorder detection leveraging the state-of-the-art object detection algorithm, YOLOv8 to detect and classify visual cues associated with specific mental disorders. To achieve accurate predictions, an integrated learning architecture based on the fusion of Convolution Neural Networks (CNNs) and Visual Transformer (ViT) models is developed to form an ensemble classifier that predicts the presence of mental illness (e.g., depression, anxiety, and other mental disorder). The overall accuracy is improved to about 81% using the proposed ensemble technique. To ensure transparency and interpretability, we integrate techniques such as Gradient-weighted Class Activation Mapping (Grad-CAM) and saliency maps to highlight the regions in the input image that significantly contribute to the model's predictions thus providing healthcare professionals with a clear understanding of the features influencing the system's decisions thereby enhancing trust and more informed diagnostic process.

Hierarchical Locality-Aware Deep Dictionary Learning for Classification

Deep dictionary learning (DDL) shows good performance in visual classification tasks. However, almost all existing DDL methods ignore the locality relationships between the input data representations and the learned dictionary atoms, and learn sub-optimal representations in the feature coding stage, which are less conducive to classification. To this end, we propose a hierarchical locality-aware deep dictionary learning (HILADLE) framework for classification, which can learn locality-constrained dictionaries at different abstract levels through hierarchical dictionary learning. The locality constraints play an important role in learning informative dictionary atoms while preserving the data structure in the original input feature space. Moreover, instead of using an identity activation function like existing DDL methods, we further boost the generalization performance of our HILADLE method with a ReLU activation function to deal with the overfitting issue caused by over-parameterization, inspired by its effectiveness in deep neural networks. Finally, the concatenation of all feature representations learned at different layers is used as input to the final classifier. We demonstrate, through an extensive set of experiments on several benchmark face recognition, image classification, and age estimation datasets, that our method is able to surpass several dictionary learning, deep dictionary learning and deep learning methods.

A Hybrid Deep Learning Approach for Epileptic Seizure Detection in EEG signals.

Early detection and proper treatment of epilepsy is essential and meaningful to those who suffer from this disease. The adoption of deep learning (DL) techniques for automated epileptic seizure detection using electroencephalography (EEG) signals has shown great potential in making the most appropriate and fast medical decisions. However, DL algorithms have high computational complexity and suffer low accuracy with imbalanced medical data in multi seizure-classification task. Motivated from the aforementioned challenges, we present a simple and effective hybrid DL approach for epileptic seizure detection in EEG signals. Specifically, first we use a K-means Synthetic minority oversampling technique (SMOTE) to balance the sampling data. Second, we integrate a 1D Convolutional Neural Network (CNN) with a Bidirectional Long Short-Term Memory (BiLSTM) network based on Truncated Backpropagation Through Time (TBPTT) to efficiently extract spatial and temporal sequence information while reducing computational complexity. Finally, the proposed DL architecture uses softmax and sigmoid classifiers at the classification layer to perform multi and binary seizure-classification tasks. In addition, the 10-fold cross-validation technique is performed to show the significance of the proposed DL approach. Experimental results using the publicly available UCI epileptic seizure recognition data set shows better performance in terms of precision, sensitivity, specificity, and F1-score over some baseline DL algorithms and recent state-of-the-art techniques.

VSFormer: Mining Correlations in Flexible View Set for Multi-view 3D Shape Understanding.

View-based methods have demonstrated promising performance in 3D shape understanding. However, they tend to make strong assumptions about the relations between views or learn the multi-view correlations indirectly, which limits the flexibility of exploring inter-view correlations and the effectiveness of target tasks. To overcome the above problems, this paper investigates flexible organization and explicit correlation learning for multiple views. In particular, we propose to incorporate different views of a 3D shape into a permutation-invariant set, referred to as View Set, which removes rigid relation assumptions and facilitates adequate information exchange and fusion among views. Based on that, we devise a nimble Transformer model, named VSFormer, to explicitly capture pairwise and higher-order correlations of all elements in the set. Meanwhile, we theoretically reveal a natural correspondence between the Cartesian product of a view set and the correlation matrix in the attention mechanism, which supports our model design. Comprehensive experiments suggest that VSFormer has better flexibility, efficient inference efficiency and superior performance. Notably, VSFormer reaches state-of-the-art results on various 3d recognition datasets, including ModelNet40, ScanObjectNN and RGBD. It also establishes new records on the SHREC'17 retrieval benchmark. The code and datasets are available at https://github.com/auniquesun/VSFormer.

Research on cross-modal emotion recognition based on multi-layer semantic fusion.

Multimodal emotion analysis involves the integration of information from various modalities to better understand human emotions. In this paper, we propose the Cross-modal Emotion Recognition based on multi-layer semantic fusion (CM-MSF) model, which aims to leverage the complementarity of important information between modalities and extract advanced features in an adaptive manner. To achieve comprehensive and rich feature extraction from multimodal sources, considering different dimensions and depth levels, we design a parallel deep learning algorithm module that focuses on extracting features from individual modalities, ensuring cost-effective alignment of extracted features. Furthermore, a cascaded cross-modal encoder module based on Bidirectional Long Short-Term Memory (BILSTM) layer and Convolutional 1D (ConV1d) is introduced to facilitate inter-modal information complementation. This module enables the seamless integration of information across modalities, effectively addressing the challenges associated with signal heterogeneity. To facilitate flexible and adaptive information selection and delivery, we design the Mask-gated Fusion Networks (MGF-module), which combines masking technology with gating structures. This approach allows for precise control over the information flow of each modality through gating vectors, mitigating issues related to low recognition accuracy and emotional misjudgment caused by complex features and noisy redundant information. The CM-MSF model underwent evaluation using the widely recognized multimodal emotion recognition datasets CMU-MOSI and CMU-MOSEI. The experimental findings illustrate the exceptional performance of the model, with binary classification accuracies of 89.1% and 88.6%, as well as F1 scores of 87.9% and 88.1% on the CMU-MOSI and CMU-MOSEI datasets, respectively. These results unequivocally validate the effectiveness of our approach in accurately recognizing and classifying emotions.

A Gating Model for Bias Calibration in Generalized Zero-shot Learning.

Generalized zero-shot learning (GZSL) aims at training a model that can generalize to unseen class data by only using auxiliary information. One of the main challenges in GZSL is a biased model prediction toward seen classes caused by overfitting on only available seen class data during training. To overcome this issue, we propose a two-stream autoencoder-based gating model for GZSL. Our gating model predicts whether the query data is from seen classes or unseen classes, and utilizes separate seen and unseen experts to predict the class independently from each other. This framework avoids comparing the biased prediction scores for seen classes with the prediction scores for unseen classes. In particular, we measure the distance between visual and attribute representations in the latent space and the cross-reconstruction space of the autoencoder. These distances are utilized as complementary features to characterize unseen classes at different levels of data abstraction. Also, the two-stream autoencoder works as a unified framework for the gating model and the unseen expert, which makes the proposed method computationally efficient. We validate our proposed method in four benchmark image recognition datasets. In comparison with other state-of-the-art methods, we achieve the best harmonic mean accuracy in SUN and AWA2, and the second best in CUB and AWA1. Furthermore, our base model requires at least 20% less number of model parameters than state-of-the-art methods relying on generative models.

Explicable Fine-Grained Aircraft Recognition Via Deep Part Parsing Prior Framework for High-Resolution Remote Sensing Imagery.

Aircraft recognition is crucial in both civil and military fields, and high-spatial resolution remote sensing has emerged as a practical approach. However, existing data-driven methods fail to locate discriminative regions for effective feature extraction due to limited training data, leading to poor recognition performance. To address this issue, we propose a knowledge-driven deep learning method called the explicable aircraft recognition framework based on a part parsing prior (APPEAR). APPEAR explicitly models the aircraft's rigid structure as a pixel-level part parsing prior, dividing it into five parts: 1) the nose; 2) left wing; 3) right wing; 4) fuselage; and 5) tail. This fine-grained prior provides reliable part locations to delineate aircraft architecture and imposes spatial constraints among the parts, effectively reducing the search space for model optimization and identifying subtle interclass differences. A knowledge-driven aircraft part attention (KAPA) module uses this prior to achieving a geometric-invariant representation for identifying discriminative features. Part features are generated by part indexing in a specific order and sequentially embedded into a compact space to obtain a fixed-length representation for each part, invariant to aircraft orientation and scale. The part attention module then takes the embedded part features, adaptively reweights their importance to identify discriminative parts, and aggregates them for recognition. The proposed APPEAR framework is evaluated on two aircraft recognition datasets and achieves superior performance. Moreover, experiments with few-shot learning methods demonstrate the robustness of our framework in different tasks. Ablation analysis illustrates that the fuselage and wings of the aircraft are the most effective parts for recognition.

Transformer-Based Self-Supervised Multimodal Representation Learning for Wearable Emotion Recognition

Recently, wearable emotion recognition based on peripheral physiological signals has drawn massive attention due to its less invasive nature and its applicability in real-life scenarios. However, how to effectively fuse multimodal data remains a challenging problem. Moreover, traditional fully-supervised based approaches suffer from overfitting given limited labeled data. To address the above issues, we propose a novel self-supervised learning (SSL) framework for wearable emotion recognition, where efficient multimodal fusion is realized with temporal convolution-based modality-specific encoders and a transformer-based shared encoder, capturing both intra-modal and inter-modal correlations. Extensive unlabeled data is automatically assigned labels by five signal transforms, and the proposed SSL model is pre-trained with signal transformation recognition as a pretext task, allowing the extraction of generalized multimodal representations for emotion-related downstream tasks. For evaluation, the proposed SSL model was first pre-trained on a large-scale self-collected physiological dataset and the resulting encoder was subsequently frozen or fine-tuned on three public supervised emotion recognition datasets. Ultimately, our SSL-based method achieved state-of-the-art results in various emotion classification tasks. Meanwhile, the proposed model was proved to be more accurate and robust compared to fully-supervised methods on low data regimes.

MEN: Mutual Enhancement Networks for Sign Language Recognition and Education.

The performance of existing sign language recognition approaches is typically limited by the scale of training data. To address this issue, we propose a mutual enhancement network (MEN) for joint sign language recognition and education. First, a sign language recognition system built upon a spatial-temporal network is proposed to recognize the semantic category of a given sign language video. Besides, a sign language education system is developed to detect the failure modes of learners and further guide them to sign correctly. Our theoretical contribution lies in formulating the above two systems as an estimation-maximization (EM) framework, which can progressively boost each other. The recognition system could become more robust and accurate with more training data collected by the education system, while the education system could guide the learners to sign more precisely, benefiting from the hand shape analysis module of the recognition system. Experimental results on three large-scale sign language recognition datasets validate the superiority of the proposed framework.

Plant disease recognition datasets in the age of deep learning: challenges and opportunities.

Although plant disease recognition has witnessed a significant improvement with deep learning in recent years, a common observation is that current deep learning methods with decent performance tend to suffer in real-world applications. We argue that this illusion essentially comes from the fact that current plant disease recognition datasets cater to deep learning methods and are far from real scenarios. Mitigating this illusion fundamentally requires an interdisciplinary perspective from both plant disease and deep learning, and a core question arises. What are the characteristics of a desired dataset? This paper aims to provide a perspective on this question. First, we present a taxonomy to describe potential plant disease datasets, which provides a bridge between the two research fields. We then give several directions for making future datasets, such as creating challenge-oriented datasets. We believe that our paper will contribute to creating datasets that can help achieve the ultimate objective of deploying deep learning in real-world plant disease recognition applications. To facilitate the community, our project is publicly available at https://github.com/xml94/PPDRD with the information of relevant public datasets.

MFE-transformer: Adaptive English text named entity recognition method based on multi-feature extraction and transformer

English text named entity recognition aims to alleviate the problem of insufficient labeling data in the target domain. Existing methods usually use feature representation or model parameter sharing to realize cross-domain transfer of entity recognition capability, but there is still a lack of full utilization of structured knowledge in text sequences. Therefore, this paper proposes an adaptive English named text entity recognition method based on multi-feature extraction and transformer. Firstly, a bidirectional long term memory conditional random field entity recognition model based on BERT pre-trained language model is constructed on a generic domain dataset. In the training process, the weights of two character vectors of text words are dynamically calculated and combined, which makes the model make full use of the information in the character granularity, and the parts-of-speech information and block analysis are added as additional features. The word vectors, character-level features and additional features are spliced into the BiLSTM-CRF neural network model for training. Finally, experiments are carried out on five English datasets and specific cross-domain named entity recognition datasets respectively. The results show that the average performance of the proposed model is improved by 0.43% and 1.47% compared with the current cross-domain model, indicating that the structured knowledge in feature representation can effectively improve the entity recognition capability of the target domain.

B-HAR: An Open-Source Baseline Framework for In-Depth Study of Human Activity Recognition Datasets and Workflows

B-HAR: An Open-Source Baseline Framework for In-Depth Study of Human Activity Recognition Datasets and Workflows

Variant Parallelism: Lightweight Deep Convolutional Models for Distributed Inference on IoT Devices

Two major techniques are commonly used to meet real-time inference limitations when distributing models across resource-constrained IoT devices: (1) model parallelism () and (2) class parallelism (). In, transmitting bulky intermediate data (orders of magnitude larger than input) between devices imposes huge communication overhead. Although solves this problem, it has limitations on the number of sub-models. In addition, both solutions are fault intolerant, an issue when deployed on edge devices. We propose variant parallelism (), an ensemble-based deep learning distribution method where different variants of a main model are generated and can be deployed on separate machines. We design a family of lighter models around the original model, and train them simultaneously to improve accuracy over single models. Our experimental results on six common mid-sized object recognition datasets demonstrate that our models can have 5.87.1× fewer parameters, 4.331× fewer multiply-accumulations (MACs), and 2.513.2× less response time on atomic inputs compared to MobileNetV2 while achieving comparable or higher accuracy. Our technique easily generates several variants of the base architecture. Each variant returns only 2k outputs 1≤k≤#classes2, representing Topk classes, instead of tons of floating point values required in . Since each variant provides a full-class prediction, our approach maintains higher availability compared with and in presence of failure.