Closed-set Recognition Research Articles

Adversarial Kinetic Prototype Framework for Open Set Recognition.

Due to the complexity of real-world applications, open set recognition is often more practical than closed set recognition. Compared with closed set recognition, open set recognition needs not only to recognize known classes but also to identify unknown classes. Different from most of the current methods, we proposed three novel frameworks with kinetic pattern to address the open set recognition problems, and they are kinetic prototype framework (KPF), adversarial KPF (AKPF), and an upgraded version of the AKPF, AKPF++. First, KPF introduces a novel kinetic margin constraint radius, which can improve the compactness of the known features to increase the robustness for the unknowns. Based on KPF, AKPF can generate adversarial samples and add these samples into the training phase, which can improve the performance with the adversarial motion of the margin constraint radius. Compared with AKPF, AKPF++ further improves the performance by adding more generated data into the training phase. Extensive experimental results on various benchmark datasets indicate that the proposed frameworks with kinetic pattern are superior to other existing approaches and achieve the state-of-the-art performance.

Spatio-temporal representation learning enhanced source cell-phone recognition from speech recordings

The existing source cell-phone recognition method lacks the long-term feature characterization of the source device, resulting in an inaccurate representation of the source cell-phone related features, which leads to insufficient recognition accuracy. In this paper, we propose a source cell-phone recognition method based on spatio-temporal representation learning, which includes two main parts: extraction of sequential Gaussian mean matrix features and construction of a recognition model based on spatio-temporal representation learning. In the feature extraction part, based on the analysis of the time-series representation of recording source signals, we extract sequential Gaussian mean matrix with long-term and short-term representation ability by using the sensitivity of Gaussian mixture model to data distribution. In the model construction part, we design a structured spatio-temporal representation learning network C3D-BiLSTM to fully characterize the spatio-temporal information, combine 3D convolutional network and bidirectional long short-term memory network for short-term spectral information and long-time fluctuation information representation learning, and achieve accurate recognition of cell-phones by fusing spatio-temporal feature information of recording source signals. The method achieves an average accuracy of 99.03% for the closed-set recognition of 45 cell-phones under the CCNU_Mobile dataset and 98.18% in small sample size experiments, with recognition performance better than the existing state-of-the-art methods. The experimental results show that the method exhibits excellent recognition performance in multi-class cell-phones recognition. Our codes are available at https://github.com/CCNUZFW/STRLESCR-SGMM.

Improving Known-Unknown Cattle's Face Recognition for Smart Livestock Farm Management.

Accurate identification of individual cattle is of paramount importance in precision livestock farming, enabling the monitoring of cattle behavior, disease prevention, and enhanced animal welfare. Unlike human faces, the faces of most Hanwoo cattle, a native breed of Korea, exhibit significant similarities and have the same body color, posing a substantial challenge in accurately distinguishing between individual cattle. In this study, we sought to extend the closed-set scope (only including identifying known individuals) to a more-adaptable open-set recognition scenario (identifying both known and unknown individuals) termed Cattle's Face Open-Set Recognition (CFOSR). By integrating open-set techniques to enhance the closed-set accuracy, the proposed method simultaneously addresses the open-set scenario. In CFOSR, the objective is to develop a trained model capable of accurately identifying known individuals, while effectively handling unknown or novel individuals, even in cases where the model has been trained solely on known individuals. To address this challenge, we propose a novel approach that integrates Adversarial Reciprocal Points Learning (ARPL), a state-of-the-art open-set recognition method, with the effectiveness of Additive Margin Softmax loss (AM-Softmax). ARPL was leveraged to mitigate the overlap between spaces of known and unknown or unregistered cattle. At the same time, AM-Softmax was chosen over the conventional Cross-Entropy loss (CE) to classify known individuals. The empirical results obtained from a real-world dataset demonstrated the effectiveness of the ARPL and AM-Softmax techniques in achieving both intra-class compactness and inter-class separability. Notably, the results of the open-set recognition and closed-set recognition validated the superior performance of our proposed method compared to existing algorithms. To be more precise, our method achieved an AUROC of 91.84 and an OSCR of 87.85 in the context of open-set recognition on a complex dataset. Simultaneously, it demonstrated an accuracy of 94.46 for closed-set recognition. We believe that our study provides a novel vision to improve the classification accuracy of the closed set. Simultaneously, it holds the potential to significantly contribute to herd monitoring and inventory management, especially in scenarios involving the presence of unknown or novel cattle.

OpenMix+: Revisiting Data Augmentation for Open Set Recognition

Open set recognition requires models to recognize samples of known classes learned in the training set while reject unknowns not learned. Compared with the structural risk minimization theory for closed-set problems, structural risk in open set tasks remains rarely explored. In this paper, we point out that balancing between structural risk and open space risk is crucial for open set recognition, and re-formalize it as open set structural risk. This brings a new view towards the general relationship between closed set recognition and open set recognition against the common intuition, which argues that a good closed set classifier always benefits for open set recognition. Specifically, we theoretically and experimentally show that recent mix-based data augmentation methods are aggressive closed set regularization methods, which reduce structural risk at cost of sacrificing open space risk. Besides, we show that existing negative data augmentation designed for open space risk reduction also ignore the trade-off problem between structural risk and open space risk, which limits their performance. We propose an efficient negative data augmentation strategy named self-mix and a corresponding method named OpenMix. OpenMix generates high-quality negative samples by mixing samples themselves, which can take care of both risks simultaneously. When combining OpenMix with conservative closed set regularization methods to form OpenMix+, models can achieve lower open set structural risk. Extensive experiments validate the superiority of OpenMix and OpenMix+ in terms of both effectiveness and universality.

Open Set Bioacoustic Signal Classification based on Class Anchor Clustering with Closed Set Unknown Bioacoustic Signals.

Recently, deep learning-driven studies have been introduced for bioacoustic signal classification. Most of them, however, have the limitation that the input of the classifier needs to match with a trained label which is known as closed set recognition (CSR). To this end, the classifier trained by CSR would not cover a real stream task since the input of the classifier has so many variations. To combat real-world tasks, open set recognition (OSR) has been developed. In OSR, randomly collected inputs are fed to the classifier and the classifier predicts target classes and Unknown class. However, this OSR has been spotlighted in the studies of computer vision and speech domains while the domain of bioacoustic signal is less developed. Especially, to our best knowledge, OSR for animal sound classification has not been studied. This paper proposes a novel method for open set bioacoustic signal classification based on Class Anchored Clustering (CAC) loss with closed set unknown bioacoustic signals. To use the closed set unknown signals for training, a total of n +1 classes are used by adding one additional Unknown class to n target classes, and n +1 cross-entropy loss is added to the CAC loss. To evaluate the proposed method, we build an animal sound dataset that includes 101 species of sounds and compare its performance with baseline methods. In the experiments, our proposed method shows higher performance than other baseline methods in the area under the receiver operating curve for detecting target class and unknown class, the classification accuracy of open set signals, and classification accuracy for target classes. As a result, the closed set class samples are well classified while the open set unknown class can be also recognized with high accuracy at the same time.

Open-Set Event Recognition Model Using 1-D RL-CNN With OpenMax Algorithm for Distributed Optical Fiber Vibration Sensing System

Distributed optical fiber vibration sensing system (DVS) base on phase-sensitive OTDR is widely used for its simple structure and high sensitivity. Signal recognition is crucial for DVS because it can help to classify the different types of vibration events. Deep learning provides accurate event classification and can automatically extract features according to sample distribution. However, almost all current methods focus on closed set recognition, which misclassifies unknown events into known categories thus reduces the recognition accuracy of sensing system. In this paper, we propose a novel open set event recognition model based on one-dimension residual learning convolution neural network (1-D RL-CNN) with OpenMax algorithm for DVS, which is capable of processing the signals of known and unknown categories. The experimental results show that the proposed recognition model improves the classification accuracy greatly compared with conventional 1-D CNN signal classification method. The overall open-set classification accuracy of 1-D RL-CNN with OpenMax is 91.19%, which is improved of 18.47% and 7.57% compared with 1-D CNN with SoftMax and 1-D CNN with OpenMax.

A Novel Method for Recognizing Space Radiation Sources Based on Multi-Scale Residual Prototype Learning Network

As a basic task and key link of space situational awareness, space target recognition has become crucial in threat analysis, communication reconnaissance and electronic countermeasures. Using the fingerprint features carried by the electromagnetic signal to recognize is an effective method. Because traditional radiation source recognition technologies are difficult to obtain satisfactory expert features, automatic feature extraction methods based on deep learning have become popular. Although many deep learning schemes have been proposed, most of them are only used to solve the inter-class separable problem and ignore the intra-class compactness. In addition, the openness of the real space may invalidate the existing closed-set recognition methods. In order to solve the above problems, inspired by the application of prototype learning in image recognition, we propose a novel method for recognizing space radiation sources based on a multi-scale residual prototype learning network (MSRPLNet). The method can be used for both the closed- and open-set recognition of space radiation sources. Furthermore, we also design a joint decision algorithm for an open-set recognition task to identify unknown radiation sources. To verify the effectiveness and reliability of the proposed method, we built a set of satellite signal observation and receiving systems in a real external environment and collected eight Iridium signals. The experimental results show that the accuracy of our proposed method can reach 98.34% and 91.04% for the closed- and open-set recognition of eight Iridium targets, respectively. Compared to similar research works, our method has obvious advantages.

Spatial location constraint prototype loss for open set recognition

One of the challenges in pattern recognition is open set recognition. Compared with closed set recognition, open set recognition needs to reduce not only the empirical risk, but also the open space risk, and how to reduce the open space risk is the key of open set recognition. Compared with the previous work, this paper analyzes the distribution rules of the known and unknown features, which is highly related to the open space risk. Then, this paper proposes the matching theory to explain the origin of the open space risk. On this basis, the spatial location constraint prototype loss function is proposed to reduce both risks simultaneously. Extensive experiments on multiple benchmark datasets and many visualization results verify the validity of the proposed matching theory and the effectiveness of the proposed method.

Radar HRRP Open Set Target Recognition Based on Closed Classification Boundary

Radar automatic target recognition based on high-resolution range profile (HRRP) has become a research hotspot in recent years. The current works mainly focus on closed set recognition, where all the test samples are assigned to the training classes. However, radar may capture many unknown targets in practical applications, and most current methods are incapable of identifying the unknown targets as the ’unknown’. Therefore, open set recognition is proposed to solve this kind of recognition task. This paper analyzes the basic classification principle of both recognitions and makes sure that determining the closed classification boundary is the key to addressing open set recognition. To achieve this goal, this paper proposes a novel boundary detection algorithm based on the distribution balance property of k-nearest neighbor objects, which can be used to realize the identification of the known and unknown targets simultaneously by detecting the boundary of the known classes. Finally, extensive experiments based on measured HRRP data have demonstrated that the proposed algorithm is indeed helpful to greatly improve the open set performance by determining the closed classification boundary of the known classes.

Radar HRRP Open Set Recognition Based on Extreme Value Distribution

Radar automatic target recognition (RATR) based on high resolution range profile (HRRP) has attracted more attention in recent years. In fact, the actual application environment of RATR is open set environment rather than closed set environment. However, previous works mainly focus on closed set recognition, which classifies the known classes by dividing hyper-planes in the feature space, and it will cause classification errors in open set environment. Therefore, open set recognition is proposed to solve this problem, which needs to determine a closed classification boundary for the identification of the known and unknown targets simultaneously. To accomplish this purpose, this paper proposes and proves the extreme value boundary theorem, which demonstrates that the maximum distance from the known features to the cluster center follows the generalized extreme value distribution. According to the proposed theorem, the closed classification boundary of the cluster is easily determined to distinguish between the known and unknown classes. Finally, extensive experiments on measured HRRP data verify the validity of the proposed theorem and the effectiveness of the proposed method.

A Research of Gas Open-Set Identification Based on Data Augmentation Algorithm

Significant progress has been made in convolutional neural networks (CNN) based gas recognition. However, existing electronic nose (e-Nose) algorithms all use the closed-set assumption that the test and training samples are in the same label space and can only detect objects of known classes. However, in realistic scenarios, collecting data and training for every possible gas would waste much resource. Open-set identification aims to actively reject samples from unknown classes by reducing the intra-class spacing and, thus, not misclassifying them as known classes. In this study, we propose a data preprocessing method to enhance the performance of closed-set recognition by augmenting the eigenvalues of each gas. We then implement the open-set recognition task for gases using an open-set recognition model. These methods contribute to improved recognition accuracy for gases and provide an effective means of handling unknown class samples. Experimental results show that our approach can identify unknown samples well while maintaining accuracy for available classes.

MoEP-AE: Autoencoding Mixtures of Exponential Power Distributions for Open-Set Recognition

Open-set recognition aims to identify unknown classes while maintaining classification performance on known classes and has attracted increasing attention in the pattern recognition field. However, how to learn effective feature representations whose distributions are usually complex for classifying both known-class and unknown-class samples when only the known-class samples are available for training is an ongoing issue in open-set recognition. In contrast to methods implementing a single Gaussian, a mixture of Gaussians (MoG), or multiple MoGs, we propose a novel autoencoder that learns feature representations by modeling them as mixtures of exponential power distributions (MoEPs) in latent spaces called MoEP-AE. The proposed autoencoder considers that many real-world distributions are sub-Gaussian or super-Gaussian and can thus be represented by MoEPs rather than a single Gaussian or an MoG or multiple MoGs. We design a differentiable sampler that can sample from an MoEP to guarantee that the proposed autoencoder is trained effectively. Furthermore, we propose an MoEP-AE-based method for open-set recognition by introducing a discrimination strategy, where the MoEP-AE is used to model the distributions of the features extracted from the input known-class samples by minimizing a designed loss function at the training stage, called MoEP-AE-OSR. Extensive experimental results in both standard-dataset and cross-dataset settings demonstrate that the MoEP-AE-OSR method outperforms 14 existing open-set recognition methods in most cases in both open-set recognition and closed-set recognition tasks.

Can Closed-Set Word Recognition Differentially Assess Vowel and Consonant Perception for School-Age Children With and Without Hearing Loss?

Vowels and consonants play different roles in language acquisition and speech recognition, yet standard clinical tests do not assess vowel and consonant perception separately. As a result, opportunities for targeted intervention may be lost. This study evaluated closed-set word recognition tests designed to rely predominantly on either vowel or consonant perception and compared results with sentence recognition scores. Participants were children (5-17 years of age) and adults (18-38 years of age) with normal hearing and children with sensorineural hearing loss (7-17 years of age). Speech reception thresholds (SRTs) were measured in speech-shaped noise. Children with hearing loss were tested with their hearing aids. Word recognition was evaluated using a three-alternative forced-choice procedure, with a picture-pointing response; monosyllabic target words varied with respect to either consonant or vowel content. Sentence recognition was evaluated for low- and high-probability sentences. In a subset of conditions, stimuli were low-pass filtered to simulate a steeply sloping hearing loss in participants with normal hearing. Children's SRTs improved with increasing age for words and sentences. Low-pass filtering had a larger effect for consonant-variable words than vowel-variable words for both children and adults with normal hearing, consistent with the greater high-frequency content of consonants. Children with hearing loss tested with hearing aids tended to perform more poorly than age-matched children with normal hearing, particularly for sentence recognition, but consonant- and vowel-variable word recognition did not appear to be differentially affected by the amount of high- and low-frequency hearing loss. Closed-set recognition of consonant- and vowel-variable words appeared to differentially evaluate vowel and consonant perception but did not vary by configuration of hearing loss in this group of pediatric hearing aid users. Word scores obtained in this manner do not fully characterize the auditory abilities necessary for open-set sentence recognition, but they do provide a general estimate.

An Automated Framework Based on Deep Learning for Shark Recognition

The recent progress in deep learning has given rise to a non-invasive and effective approach for animal biometrics. These modern techniques allow researchers to track animal individuals on a large-scale image database. Typical approaches are suited to a closed-set recognition problem, which is to identify images of known objects only. However, such approaches are not scalable because they mis-classify images of unknown objects. To recognize the images of unknown objects as ‘unknown’, a framework should be able to deal with the open set recognition scenario. This paper proposes a fully automatic, vision-based identification framework capable of recognizing shark individuals including those that are unknown. The framework first detects and extracts the shark from the original image. After that, we develop a deep network to transform the extracted image to an embedding vector in latent space. The proposed network consists of the Visual Geometry Group-UNet (VGG-UNet) and a modified Visual Geometry Group-16 (VGG-16) network. The VGG-UNet is utilized to detect shark bodies, and the modified VGG-16 is used to learn embeddings of shark individuals. For the recognition task, our framework learns a decision boundary using a one-class support vector machine (OSVM) for each shark included in the training phase using a few embedding vectors belonging to them, then it determines whether a new shark image is recognized as belonging to a known shark individual. Our proposed network can recognize shark individuals with high accuracy and can effectively deal with the open set recognition problem with shark images.

LUNA: Localizing Unfamiliarity Near Acquaintance for Open-Set Long-Tailed Recognition

The predefined artificially-balanced training classes in object recognition have limited capability in modeling real-world scenarios where objects are imbalanced-distributed with unknown classes. In this paper, we discuss a promising solution to the Open-set Long-Tailed Recognition (OLTR) task utilizing metric learning. Firstly, we propose a distribution-sensitive loss, which weighs more on the tail classes to decrease the intra-class distance in the feature space. Building upon these concentrated feature clusters, a local-density-based metric is introduced, called Localizing Unfamiliarity Near Acquaintance (LUNA), to measure the novelty of a testing sample. LUNA is flexible with different cluster sizes and is reliable on the cluster boundary by considering neighbors of different properties. Moreover, contrary to most of the existing works that alleviate the open-set detection as a simple binary decision, LUNA is a quantitative measurement with interpretable meanings. Our proposed method exceeds the state-of-the-art algorithm by 4-6% in the closed-set recognition accuracy and 4% in F-measure under the open-set on the public benchmark datasets, including our own newly introduced fine-grained OLTR dataset about marine species (MS-LT), which is the first naturally-distributed OLTR dataset revealing the genuine genetic relationships of the classes.

Open-set gas recognition: A case-study based on an electronic nose dataset

Electronic nose (E-Nose) has been widely used in detection and classification of gases. The learning models of traditional E-Noses are generally limited in closed-set environment: the training and test samples share the same label spaces. However, a more challenging and realistic scenario of E-Noses is open-set learning, where the test samples contains classes unseen during the model training. This study investigated the possibility of open-set learning models for the recognition and classification of gases based on a public electronic nose datasets. The dataset includes the response of a 72-channels MOS sensor array on 10 gaseous substances. The original data was preprocessed by two methods: one is to manually extract features from the response curve of each sample, and the other is to down-sample the original sample into a matrix. Then multilayer perceptron (MLP) and convolution neural network (CNN) were used to extract the feature vectors of the data processed by the two processing methods respectively. The performance of four different open-set recognition models, including softmax threshold (ST), OpenMax, extreme value machine (EVM) and class anchor clustering (CAC), was compared based on the feature vectors obtained from two neural networks. To understand the effect of sensor drift on the models, we also validated the models on a commonly used sensor drift dataset. The results demonstrated that for the open-set detection task, the CNN-based CAC (CAC-CNN) outperformed the other methods. For the closed-set recognition task, the CNN-based classification model achieved higher accuracy. On sensor drift dataset, the performance of open-set recognition models has decreased a lot, and it seems that drift has a large negative impact on the open-set gas recognition.

Spatial Location Constraint Prototype Loss for Open Set Recognition

One of the challenges in pattern recognition is open set recognition. Compared with closed set recognition, open set recognition needs to reduce not only the empirical risk, but also the open space risk, and the reduction of these two risks corresponds to classifying the known classes and identifying the unknown classes respectively. How to reduce the open space risk is the key of open set recognition. This paper explores the origin of the open space risk by analyzing the distribution of known and unknown classes features. On this basis, the spatial location constraint prototype loss function is proposed to reduce the two risks simultaneously. Extensive experiments on multiple benchmark datasets and many visualization results indicate that our methods is superior to most existing approaches.

Spatial Location Constraint Prototype Loss for Open Set Recognition

One of the challenges in pattern recognition is open set recognition. Compared with closed set recognition, open set recognition needs to reduce not only the empirical risk, but also the open space risk, and the reduction of these two risks corresponds to classifying the known classes and identifying the unknown classes respectively. How to reduce the open space risk is the key of open set recognition. This paper explores the origin of the open space risk by analyzing the distribution of known and unknown classes features. On this basis, the spatial location constraint prototype loss function is proposed to reduce the two risks simultaneously. Extensive experiments on multiple benchmark datasets and many visualization results indicate that our methods is superior to most existing approaches.

Surrounding Prototype Loss for Radar HRRP Open Set Target Recognition

The open set recognition model can identify the known and unknown samples simultaneously. In the radar automatic target recognition application, open set recognition meets better practical requirements than closed set recognition. Theoretically, an overlap usually exists between the known and unknown features, which makes it difficult for the model to identify unknown samples. Therefore, we explore the distribution of the known and unknown features, and find that the unknown features are usually smaller and closer to the center region in the feature space than the known features. Based on this phenomenon, we propose a novel loss function that improves the open set recognition performance by controlling the known features distributed to the surrounding area of the feature space. In addition, extensive experiments are carried out on measured HRRP data. Thus, we verify the effectiveness of the proposed method.

Printed Circuit Board identification using Deep Convolutional Neural Networks to facilitate recycling

In this paper, we have proposed a robust Printed Circuit Board (PCB) classification system based on computer vision and deep learning to assist sorting e-waste for recycling. We have used a public PCB dataset acquired using a conveyor belt, as well as a locally developed PCB dataset that represents challenging practical conditions such as varying lighting, orientation, distance from camera, cast shadows, view-points and different cameras/resolutions. A pre-trained EfficientNet-B3 deep learning model is utilized and retrained for use with our data in PCB classification context. Deep nets are designed for closed set recognition tasks capable of classifying only the images they have been trained for. We have extended the closed set nature of deep nets for use in our open set classification tasks which require identifying unknown PCBs apart from classifying known PCBs. We have achieved an open set average accuracy of 92.4% which is state of the art given the complexities in the datasets we worked with.