Degradation Of Recognition Performance Research Articles

Weakly Supervised Transformer for Radar Jamming Recognition

Radar jamming recognition is a key step in electronic countermeasures, and accurate and sufficient labeled samples are essential for supervised learning-based recognition methods. However, in real practice, collected radar jamming samples often have weak labels (i.e., noisy-labeled or unlabeled ones), which degrade recognition performance. Additionally, recognition performance is hindered by limitations in capturing the global features of radar jamming. The Transformer (TR) has advantages in modeling long-range relationships. Therefore, a weakly supervised Transformer is proposed to address the issues of performance degradation under weak supervision. Specifically, complementary label (CL) TR, called RadarCL-TR, is proposed to improve radar jamming recognition accuracy with noisy samples. CL learning and a cleansing module are successively utilized to detect and remove potentially noisy samples. Thus, the adverse influence of noisy samples is mitigated. Additionally, semi-supervised learning (SSL) TR, called RadarSSL-PL-TR, is proposed to boost recognition performance under unlabeled samples via pseudo labels (PLs). Network generalization is improved by training with pseudo-labeling unlabeled samples. Moreover, the RadarSSL-PL-S-TR is proposed to further promote recognition performance, where a selection module identifies reliable pseudo-labeling samples. The experimental results show that the proposed RadarCL-TR and RadarSSL-PL-S-TR outperform comparison methods in recognition accuracy by at least 7.07% and 6.17% with noisy and unlabeled samples, respectively.

Beyond the Mask: Advancements in Facial Recognition through Convolutional Neural Networks

Abstract: Facial recognition technology has gained immense popularity in recent years, yet challenges persist with face alterations and the widespread use of masks. Uncooperative individuals in real-world scenarios, such as video surveillance, often resort to masking, leading to a degradation in current face recognition performance. While extensive research has addressed face recognition under various conditions, the impact of masks has often been overlooked. This research focuses specifically on improving the accuracy of recognizing faces obscured by different masks. Our proposed approach involves initial detection of facial regions, addressing the occluded face detection problem through the Multi-Task Cascaded Convolutional Neural Network (MTCNN). Subsequently, facial feature extraction utilizes the Google Face Net embedding model, followed by a classification task executed by Support Vector Machine (SVM). Experimental results demonstrate the effectiveness of this approach in masked face recognition, with noteworthy performance even under excessive facial masks. Additionally, a comparative study enhances our understanding of the proposed method's capabilities

Improving the Performance of Automatic Lip-Reading Using Image Conversion Techniques

Variation in lighting conditions is a major cause of performance degradation in pattern recognition when using optical imaging. In this study, infrared (IR) and depth images were considered as possible robust alternatives against variations in illumination, particularly for improving the performance of automatic lip-reading. The variations due to lighting conditions were quantitatively analyzed for optical, IR, and depth images. Then, deep neural network (DNN)-based lip-reading rules were built for each image modality. Speech recognition techniques based on IR or depth imaging required an additional light source that emitted light in the IR range, along with a special camera. To mitigate this problem, we propose a method that does not use an IR/depth image directly, but instead estimates images based on the optical RGB image. To this end, a modified U-net was adopted to estimate the IR/depth image from an optical RGB image. The results show that the IR and depth images were rarely affected by the lighting conditions. The recognition rates for the optical, IR, and depth images were 48.29%, 95.76%, and 92.34%, respectively, under various lighting conditions. Using the estimated IR and depth images, the recognition rates were 89.35% and 80.42%, respectively. This was significantly higher than for the optical RGB images.

A novel transformer autoencoder for multi-modal emotion recognition with incomplete data

Multi-modal signals have become essential data for emotion recognition since they can represent emotions more comprehensively. However, in real-world environments, it is often impossible to acquire complete data on multi-modal signals, and the problem of missing modalities causes severe performance degradation in emotion recognition. Therefore, this paper represents the first attempt to use a transformer-based architecture, aiming to fill the modality-incomplete data from partially observed data for multi-modal emotion recognition (MER). Concretely, this paper proposes a novel unified model called transformer autoencoder (TAE), comprising a modality-specific hybrid transformer encoder, an inter-modality transformer encoder, and a convolutional decoder. The modality-specific hybrid transformer encoder bridges a convolutional encoder and a transformer encoder, allowing the encoder to learn local and global context information within each particular modality. The inter-modality transformer encoder builds and aligns global cross-modal correlations and models long-range contextual information with different modalities. The convolutional decoder decodes the encoding features to produce more precise recognition. Besides, a regularization term is introduced into the convolutional decoder to force the decoder to fully leverage the complete and incomplete data for emotional recognition of missing data. 96.33%, 95.64%, and 92.69% accuracies are attained on the available data of the DEAP and SEED-IV datasets, and 93.25%, 92.23%, and 81.76% accuracies are obtained on the missing data. Particularly, the model acquires a 5.61% advantage with 70% missing data, demonstrating that the model outperforms some state-of-the-art approaches in incomplete multi-modal learning.

An effective iris biometric privacy protection scheme with renewability

Iris is considered a unique, robust biometric that can be used as an authentication factor. Privacy protection of individuals’ iris templates is an essential requirement. The essential attributes needed to ensure privacy of biometrics are irreversibility, renewability, and unlinkability. Information distortion is an effective way to achieve the privacy. However, this may lead to some degradation of recognition performance. A trade-off between recognition performance and the privacy level provided by the authentication system arises when it tries to provide both simultaneously. This paper proposes a novel approach that ensures the privacy of each individual's iris by distorting iris feature information. At the meantime, this approach balances the trade-off mentioned above up to a tolerable level. The binary feature templates of iris biometrics are transformed into another irreversible binary representation by manipulating them block-wise. The results obtained against two datasets using proposed approach illustrate excessive Genuine Acceptance Rates (GAR) when the approach accepts imposters at a rate which is reduced to 0.01 %. The proposed approach satisfies the three attributes that a biometric system should fulfil, namely irreversibility, renewability, and unlinkability. The renewability attribute induces the cancellability of iris biometric template. Compared to the existing works, the proposed approach maintains Equal Error Rate (EER) at 0.18 %.

Hybrid Precision in Resistive Memory-Based Convolutional Kernel for Fault-Resilient Neuromorphic Systems

As simple convolution computation is intensively and iteratively performed to extract features from input images, cross-point arrays with resistive random access memory (RRAM) serving as a kernel weight can accelerate the relevant mathematical operations in hardware. However, considering actual RRAM characteristics, either variability or unexpected permanent failure from the filamentary switching mechanism is observed, degrading recognition performance. This study investigates the impact of fault in a conventional kernel structure, where two adjacent columns in the array represent a single weight, on feature extraction using MATLAB. First, the fault types of HfO <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$_{\textit{x}}$</tex-math> </inline-formula> -based multilevel RRAM is categorized. The results reveal that the unidirectional fault of RRAM primarily worsens the accuracy of image recognition. This is because the subtraction of negative weights from positive ones is crucial for identifying the edges of images through convolution operations. Therefore, we exploit a kernel structure, in which a single column dedicated to negative weights is located next to a matrix of positive weights. In addition, we reduce the weight precision for negative weights, while quantizing positive weights to higher bits. By mitigating the subtraction errors achieved by the kernel structure with hybrid precision, we improved fault tolerance, minimizing accuracy degradation.

UnifiedFace: A Uniform Margin Loss Function for Face Recognition

Face recognition has achieved great success due to the development of deep convolutional neural networks (DCNNs) and loss functions based on margin. However, complex DCNNs bring a large number of parameters as well as computational effort, which pose a significant challenge to resource-constrained embedded devices. Meanwhile, the popular margin-based loss functions all introduce only one type of margin and cannot further introduce a larger margin to achieve tighter classification boundary. In contrast to the common approach, we believe that additive and multiplicative margins should be used jointly to introduce larger margins from the margin perspective. Therefore, we propose a new margin-based loss function called UnifiedFace. First, we introduce an additive margin in the target angle activation function. Second, we add a multiplicative margin in the non-target angle. UnifiedFace introduces both additive and multiplicative margins, allowing for the introduction of large margins to achieve more compact intra-class variance and closer separated inter-class variance. In addition, we specifically design efficient face recognition models called GhostFaceNet for resource-constrained embedded devices. Experimental results demonstrate that UnifiedFace achieves state-of-the-art performance or performance competed with popular methods in training datasets of different sizes. UnifiedFace achieves optimal performance in models of varying complexity. Moreover, competitive results are achieved in the large-scale test set IJBB/C, especially the state-of-the-art performance achieved in TAR (FAR=1e−6). GhostFaceNet can significantly improve operational efficiency without significantly degrading recognition performance, making it ideal for embedded devices with limited resources.

A Deep Learning-Based Text Detection and Recognition Approach for Natural Scenes

In this paper, we design a natural scene text detection and recognition model based on deep learning by model construction and in-depth study of wild scene text detection and recognition. This paper proposes a scene text recognition method based on connection time classification and attention mechanism for the situation where natural scene text is challenging to recognize due to the high complexity of text and background. The method converts the text recognition problem in natural scenes into a sequence recognition problem, avoiding the drawback of overall recognition performance degradation due to the difficulty of character segmentation. At the same time, the attention mechanism introduced can reduce the network complexity and improve the recognition accuracy. The performance of the improved PSE-based text detection algorithm in this paper is tested on the curved text datasets SCUT-ctw1500 and ICDAR2017 in natural scenes for comparison. The results show that the proposed algorithm achieves 88.5%, 77%, and 81.3% in the three indexes of accuracy, recall, and F1 value, respectively, without adding the pre-training module. The algorithm can detect text in any direction well without adding the pre-training module; the improved text recognition algorithm based on CRNN in this paper is tested on the natural scene dataset ICDAR2017, and the results show that the accuracy rate reaches 94.5% under the condition of no constraint, which is a good performance.

A Contactless Palmprint Imaging System Design Using Mediapipe Hands

Palmprint has been widely used in biometric systems because of its durability and reliability. To avoid recognition performance degradation, dynamic region of interest extraction is a critical step for these systems. In this study, a low-cost contactless palmprint imaging system has been designed and a dynamic region of interest extraction method has been applied to palmprints using the MediaPipe Hands framework. Since the need for hygienic touchless systems has been realized in the post-COVID-19 pandemic world, a low-cost imaging system has been proposed to capture the user’s hand at a distance without touching any platform. The region of interest of the user's palmprints in a real-time video stream has been extracted dynamically. This study creates a paradigm for future studies on palmprint imaging. With conducted experiments, the potential of MediaPipe Hands in terms of speed and accuracy on mobile palmprint imaging applications has been realized on Raspberry Pi 4. This work demonstrates that the employed hardware and proposed hand-tracking algorithm are suitable for designing low-cost contactless palmprint imaging systems in non-controlled ambient light conditions. For recognition purposes, a database will be released soon.

A Novel System and Experimental Study for 3D Finger Multibiometrics

Fingers contain various discriminative biometric traits, such as fingerprint, finger vein, finger knuckle, finger shape, and so on, which are complementary in identity information. However, only one or a few traits are used in most current research and practical applications, while others are ignored, resulting in degraded recognition performance and vulnerability to forgery. In this paper, we make the first attempt to collect and study all biometric traits on the finger. Firstly, a novel multi-view, multi-spectral 3D finger imaging system is proposed. To the best of our knowledge, it is the first biometric imaging system capable of capturing almost all finger-based traits. We scanned numerous fingers with this 3D finger imaging system, obtaining external skin images and internal vein images from 6 different views. The proposed 3D finger reconstruction and texture mapping algorithms are then used to generate 3D finger models with skin and vein textures. Second, we establish a benchmark dataset, namely the Large-scale Finger Multi-Biometric database and benchmark for 3D Finger Biometrics (LFMB-3DFB). The LFMB-3DFB contains 695 fingers, and each finger is acquired 10 times, yielding 6 finger skin images and 6 finger vein images for a total of 83,400 images and 6,950 3D finger models. Finally, we design a more scientific and comprehensive evaluation protocol to conduct extensive experimental research and analysis on this database for both subject-independent verification and subject-independent close-set identification tasks. Comprehensive and rigorous experiments for 2D finger traits recognition, multi-view finger traits recognition, 3D finger traits recognition, and score-level fusion on LFMB-3DFB are carried out, and excellent results are achieved. The LFMB-3DFB database will be released at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/SCUT-BIP-Lab/LFMB-3DFB</uri> to promote 3D finger multi-biometric research using cutting-edge imaging techniques.

Robust graph fusion and recognition framework for fingerprint and finger‐vein

The human finger is the essential carrier of biometric features. The finger itself contains multi-modal traits, including fingerprint and finger-vein, which provides convenience and practicality for finger bi-modal fusion recognition. The scale inconsistency and feature space mismatch of finger bi-modal images are important reasons for the fusion effect. The feature extraction method based on graph structure can well solve the problem of feature space mismatch for the finger bi-modalities, and the end-to-end fusion recognition can be realised based on graph convolutional neural networks (GCNs). However, this fusion recognition strategy based on GCNs still has two urgent problems: first, lack of stable and efficient graph fusion method; second, over-smoothing problem of GCNs will lead to the degradation of recognition performance. A novel fusion method is proposed to integrate the graph features of fingerprint (FP) and finger-vein (FV). Furthermore, we analyse the inner relationship between the information transmission process and the over-smoothing problem in GCNs from an optimisation perspective, and point out that the differentiated information between neighbouring nodes decreases as the number of layers increases, which is the direct reason for the over-smoothing problem. A modified deep graph convolution neural network is proposed, aiming to alleviate the over-smoothing problem. The intuition is that the differentiated features of the nodes should be properly preserved to ensure the uniqueness of the nodes themselves. Thus, a constraint term to the objective function of the GCN is added to emphasise the differentiation features of the nodes themselves. The experimental results show that the proposed fusion method can achieve more satisfied performance in finger bi-modal biometric recognition, and the proposed constrained GCN can well alleviate the problem of over-smoothing.

A New Data Model for the Privacy Protection of Medical Images

Benefiting from the intelligent Medical Internet of Things (IoMT), the medical industry has dramatically improved its quality and productivity. The transmission of biomedical data in an open and untrusted network poses a new challenge to the privacy protection of patient information. The low processing power of IoMT limited the application of traditional encryption to protect sensitive data. In this paper, we developed a new data protection model for medical images. The model uses visual cryptography (VC) to store biomedical data in a separate database, which can transfer the sensitive data of patients simply and securely. To alleviate the degradation of biomedical recognition performance caused by VC-based noise, we further use transfer learning to train an optimized neural network. The experimental results show that this proposed method provides privacy in the IoMT environment and maintains the high accuracy of biomedical recognition.

Phase-Based Palmprint Identification With Convolutional Sparse Coding

Phase-based image matching has shown high recognition accuracy in palmprint verification. The algorithm compares a pair of palmprint images by extracting local phase features from the images and computing local correlation functions between them. A major drawback of this algorithm is its high computational cost associated with the evaluation of local correlation functions. This needs to be addressed, especially in the case of one-to-many comparisons required for palmprint identification. The problem becomes increasingly severe as the number of enrolled images increases. In this paper, we propose a novel palmprint identification algorithm with low computational complexity, which employs a sparse representation of enrolled phase features (i.e., phase templates) to evaluate local correlation functions. For this purpose, we also develop an efficient Convolutional Sparse Coding (CSC) algorithm that can derive a compact representation of phase templates. The proposed method reduces the computational cost of phase-based palmprint identification without significant degradation of recognition performance. Our experiments using public databases clearly demonstrate the advantage of the proposed method over conventional methods.

Adaptive Multiexpert Learning for Lithology Recognition

Summary Lithology recognition based on well logging plays an important role in drilling optimization and minimizing investment costs. Many existing lithology recognition methods resolve this problem by assuming that they have a balanced lithology data set (i.e., the different classes of lithology data samples have an even distribution). However, this assumption is usually not feasible in the real world, where the data sample size varies enormously between different categories of lithology and thereby leads to significantly degrading recognition performance. To address this problem, we propose adaptive multiexpert learning (AMEL) in this paper. Reweighting methods can be used to solve imbalanced distributions. However, blindly increasing the loss weights of the minority class does not create a balanced model. However, AMEL incorporates reweighting into knowledge distillation (KD) to further optimize the learning process. Our process involves dividing the training set into several groups and training various expert models. Then, we trained student models using a combination of weighted instance losses and distillation losses. Furthermore, we carried out extensive experiments on real-life log data to prove the efficiency of AMEL.

Robust children’s speech recognition in zero resource condition

User applications such as voice-based web search, online learning, and video gaming require an effective speech recognition module to take user commands. Nowadays, even children are frequently using such tools, especially for online learning and gaming. This has increased the demand for developing a noise-robust automatic speech recognition (ASR) system that can effectively transcribe children’s data under varied ambient conditions. However, automatic recognition of children’s speech is extremely challenging due to the insufficiency of data from child speakers in the majority of the languages across the world. Consequently, in this zero-resource condition, we are forced to decode children’s speech on systems trained using adults’ data. However, the acoustic mismatch between adults’ and children’s speech, such as differences in pitch, formant frequencies, and speaking-rates, leads to highly degraded recognition performance. To enhance the recognition rate under zero-resource conditions, we have explored the role of formant and duration-modification-based out-of-domain data augmentation in this paper. For that purpose, the formant frequencies of the adults’ speech data are upscaled using warping of linear predictive coding coefficients. On pooling original and formant modified adults’ speech data into training, the mismatch in formant locations is reduced leading to better recognition performance. Further improvement in recognition rate can be achieved by simultaneously modifying the duration as well as the formant frequencies of the training data. This case of out-of-domain data augmentation has also been studied in this work and found to yield added gains. In addition to data augmentation, a noise- and pitch-robust front-end acoustic feature extraction approach exploiting higher-order spectral analysis (simple and cross-bispectrum) is also proposed in this paper. The proposed features are noise-robust due to the inherent immunity of the bispectrum towards additive noises. An added advantage of bispectrum is reduced pitch sensitivity as demonstrated in this work. This, in turn, helps alleviate the aforementioned pitch-induced acoustic mismatch. The experimental evaluations presented in this paper demonstrate that the use of proposed acoustic features, as well as the out-of-domain data augmentation techniques, are highly suited for zero-resource children’s speech recognition tasks under clean and noisy conditions.

Localization-Aware Adaptive Pairwise Margin Loss for Fine-Grained Image Recognition

Fine-grained image recognition is a highly challenging problem due to subtle differences between images. There are many attempts to solve fine-grained image recognition problems using data augmentation, jointly optimizing deep metric learning. CutMix is one of the excellent data augmentation strategies which crops and merges to generate new images. However, it sometimes generates meaningless and obscured object images that degrade recognition performance. We propose a novel framework that solves the above problem and expands the CutMix leveraging localizing method. Also, we improve the recognition accuracy to joint optimizing with a pairwise margin loss using generated images from the improved CutMix. There are some images similar to the reference image among the generated images. They are generated by replacing similar parts from the reference image. Those generated images should not be located much farther than the margin value in embedding space because those generated images and a reference image have similar semantic meaning. However, the conventional margin loss can not consider those images which are located much farther than the margin. To solve this problem, we propose an additional margin loss to consider those generated images. The proposed framework consists of two stages: the part localization-aware CutMix and an adaptive pairwise margin loss. The proposed method achieves state-of-the-art performance on the CUB-200-2011, FGVC-Aircraft, Stanford Cars, and DeepFashion datasets. Furthermore, extensive experiments demonstrate that each stage improves the final performance.

Depth occlusion perception feature analysis for person re-identification

• Occlusion perception person re-identification. • Research on deep neural network framework. • Person re-identification dataset GAN generation. Person re-identification (ReID) has achieved significant improvement under the setting of matching two holistic person images. However, persons are easily occluded by the various objects and other persons in real-world scenarios, making Person ReID a challenging task. In this paper, we propose a novel method named Pose-Driven Visibility Model (PDVM) to effectively solve the degradation of recognition performance caused by occlusion. Firstly, we extract non-occluded human body features through pose estimation, pay attention to the salient features of non-human parts through self-attention mechanism, and obtains the final feature representation after the combination. Secondly, we more accurately locate person body parts by utilizing the detected human keypoints in different occlusion situations, effectively reducing the impact of unalignment and realizing better matching for persons. We implement extensive experiments on Occluded-DukeMTMC and Partial-REID. Our proposed method achieves state of the art performances which reaches 53.0% Rank-1 accuracy on Occluded-DukeMTMC dataset and ablation analysis also verify the effectiveness of our method. 2020 Elsevier Ltd. All rights reserved

Nonconvex regularizer and latent pattern based robust regression for face recognition

Nuclear norm based matrix regression (NMR) approaches have yielded encouraging recognition results in the presence of image-level noise. However, NMR methods have two shortcomings. One is that test samples are directly used in the reconstruction process, and this may degrade recognition performance, especially when the samples are severely corrupted by noise. The other is that these methods exploit the nuclear norm to characterize low-rank structural information of residual images, which may lead to suboptimal solutions. To overcome these two deficiencies, we present a robust regression model that coalesces a nonconvex regularizer with a latent pattern (NRLPR). The latent pattern is essentially obtained by removing noise from a test sample, and therefore it can be closer to the reconstructed sample than the test sample. Furthermore, the nonconvex regularizer penalizes larger values less while penalizing smaller values more, accordingly, it can efficiently approximate the rank function. Additionally, we integrate the iteratively reweighted least squares method and the alternating direction method of multipliers to devise an efficient iterative algorithm (IR-ADMM) for NRLPR. Meanwhile, a nonconvex function is used for the design of the classifier. Finally, numerous experiments demonstrate the superiority of the proposed methods over state-of-the-art approaches for handling structural and mixed noise.

Hardness-Aware Dictionary Learning: Boosting Dictionary for Recognition

Sparse representation is a powerful tool in many visual applications since images can be represented effectively and efficiently with a dictionary. Conventional dictionary learning methods usually treat each training sample equally, which would lead to the degradation of recognition performance when the samples from same category distribute dispersedly. This is because the dictionary focuses more on easy samples (known as highly clustered samples), and those hard samples (known as widely distributed samples) are easily ignored. As a result, the test samples which exhibit high dissimilarities to most of intra-category samples tend to be misclassified. To circumvent this issue, this paper proposes a simple and effective hardness-aware dictionary learning (HADL) method, which considers training samples discriminatively based on the AdaBoost mechanism. Different from learning one optimal dictionary, HADL learns a set of dictionaries and corresponding sub-classifiers jointly in an iterative fashion. In each iteration, HADL learns a dictionary and a sub-classifier, and updates the weights based on the classification errors given by current sub-classifier. Those correctly classified samples are assigned with small weights while those incorrectly classified samples are assigned with large weights. Through the iterated learning procedure, the hard samples are associated with different dictionaries. Finally, HADL combines the learned sub-classifiers linearly to form a strong classifier, which improves the overall recognition accuracy effectively. Experiments on well-known benchmarks show that HADL achieves promising classification results.

On Learning Disentangled Representations for Gait Recognition.

Gait, the walking pattern of individuals, is one of the important biometrics modalities. Most of the existing gait recognition methods take silhouettes or articulated body models as gait features. These methods suffer from degraded recognition performance when handling confounding variables, such as clothing, carrying and viewing angle. To remedy this issue, we propose a novel AutoEncoder framework, GaitNet, to explicitly disentangle appearance, canonical and pose features from RGB imagery. The LSTM integrates pose features over time as a dynamic gait feature while canonical features are averaged as a static gait feature. Both of them are utilized as classification features. In addition, we collect a Frontal-View Gait (FVG) dataset to focus on gait recognition from frontal-view walking, which is a challenging problem since it contains minimal gait cues compared to other views. FVG also includes other important variations, e.g., walking speed, carrying, and clothing. With extensive experiments on CASIA-B, USF, and FVG datasets, our method demonstrates superior performance to the SOTA quantitatively, the ability of feature disentanglement qualitatively, and promising computational efficiency. We further compare our GaitNet with state-of-the-art face recognition to demonstrate the advantages of gait biometrics identification under certain scenarios, e.g., long-distance/lower resolutions, cross viewing angles. Source code is available at http://cvlab.cse.msu.edu/project-gaitnet.html.