Open-set Scenarios Research Articles

SOAMC: A Semi-Supervised Open-Set Recognition Algorithm for Automatic Modulation Classification

Traditional automatic modulation classification methods operate under the closed-set assumption, which proves to be impractical in real-world scenarios due to the diverse nature of wireless technologies and the dynamic characteristics of wireless propagation environments. Open-set environments introduce substantial technical challenges, particularly in terms of detection effectiveness and computational complexity. To address the limitations of modulation classification and recognition in open-set scenarios, this paper proposes a semi-supervised open-set recognition approach, termed SOAMC (Semi-Supervised Open-Set Automatic Modulation Classification). The primary objective of SOAMC is to accurately classify unknown modulation types, even when only a limited subset of samples is manually labeled. The proposed method consists of three key stages: (1) A signal recognition pre-training model is constructed using data augmentation and adaptive techniques to enhance robustness. (2) Feature extraction and embedding are performed via a specialized extraction network. (3) Label propagation is executed using a graph convolutional neural network (GCN) to efficiently annotate the unlabeled signal samples. Experimental results demonstrate that SOAMC significantly improves classification accuracy, particularly in challenging scenarios with limited amounts of labeled data and high signal similarity. These findings are critical for the practical identification of complex and diverse modulation signals in real-world wireless communication systems.

SLBDetection-Net: Towards closed-set and open-set student learning behavior detection in smart classroom of K-12 education

Through effective analysis of K-12 education students’ learning behaviors in the classroom, it is possible to greatly improve the interaction between teaching and learning, thereby enhancing the quality of education. However, current traditional analysis of student classroom behavior focuses on closed-set behavior detection in a single scenario. For complex and open real classroom environments, the challenge lies in obtaining meaningful representations of behaviors in small and densely populated complex scenes, while also achieving good performance in both closed-set and open-set environments. To address these challenges, this study introduces a novel method for detecting student learning behavior in both closed-set and open-set scenarios, termed SLBDetection-Net. This method focuses on accurately capturing learning behavior representation, with a specific emphasis on Multi-scale Focusing Key Information (MFKI). The study begins with designing a Learning Behavior-aware Attention (LBA) mechanism, dedicated to extracting key features of learning behaviors and capturing the complex characteristics of targets across different scales. Building on this attention mechanism, a backbone network feature encoder, LBA-Swin Transformer Block, is constructed to form the comprehensive SLBDetection-Net. The effectiveness of SLBDetection-Net is validated through rigorous testing and evaluation on real classroom scenario data of K-12 education, comparing its performance with the State-of-the-Art (SOTA) methods. The results demonstrate that SLBDetection-Net achieves a mean Average Precision (mAP) of 96.4% on the ClaBehavior dataset and 85.9% on the SCB dataset. These findings underscore the method’s significant advantages in enhancing detection precision and efficiency in both closed-set and open-set scenarios, thereby expanding the application scope of educational assessment frameworks. The source code of this study is publicly available at https://github.com/CCNUZFW/SLBDetection-Net.

Feature-semantic augmentation network for few-shot open-set recognition

Few-shot open-set recognition (FSOR) represents a relatively underexplored area of research. The primary challenge encountered by FSOR methods lies in recognizing known classes while simultaneously rejecting unknown classes utilizing only limited samples. Current FSOR methods predominantly rely on the visual information extracted from images to establish class representations, aiming to derive distinguishable classification scores for both known and unknown classes. However, these methods often overlook the benefits of leveraging semantic information derived from class names associated with images, which could provide valuable auxiliary learning insights. This study introduces a feature-semantic augmentation network to improve FSOR performance utilizing multimodal information. Specifically, we augment the class-specific features of closed-set prototypes by integrating visual and textual features from known class names across both local and global feature spaces. To facilitate prototype learning, We introduce a refinement and fusion module. Among these, the former leverages the similarity between prototype and target features at both channel and spatial dimensions to calibrate targets relative to their relevant prototypes. Meanwhile, the latter employs additional classification targets generated by the fusion module to provide learning sources from different classes. Experimental results on various few-shot learning benchmarks show that the proposed method significantly outperforms current state-of-the-art methods across both closed- and open-set scenarios.

Dynamic Attribute-guided Few-shot Open-set Network for medical image diagnosis

The scarcity of data on rare diseases poses a significant challenge to the development of diagnostic systems. While few-shot learning (FSL) offers promise in low-data regimes, it often struggles in open-set scenarios, failing to identify unknown diseases. In this paper, we introduce a Dynamic Attribute-guided Few-shot Open-set Network (DAFON), representing the first effort to simultaneously address closed-set classification and open-set recognition in rare disease diagnosis. To alleviate incomprehensive category knowledge stemming from data scarcity, we propose a Global Attribute Generator to create attributes and produce image attribute activations for closed-set data as auxiliary information. An attribute space is then constructed and employed to generate the pseudo open-set attribute activations using a designed Open-set Data Sampler. By incorporating closed-set and open-set attribute activations as conditions, we propose a Dynamic Attribute Guided Alignment module to align feature space with attribute space, so as to derive feature space with intra-class compactness for closed-set and open-set classes. Our DAFON achieves state-of-the-art performance on two public medical image datasets, demonstrating its effectiveness for FSOSR in medical image diagnosis.

Open-Set Recognition of Individual Cows Based on Spatial Feature Transformation and Metric Learning.

The automated recognition of individual cows is foundational for implementing intelligent farming. Traditional methods of individual cow recognition from an overhead perspective primarily rely on singular back features and perform poorly for cows with diverse orientation distributions and partial body visibility in the frame. This study proposes an open-set method for individual cow recognition based on spatial feature transformation and metric learning to address these issues. Initially, a spatial transformation deep feature extraction module, ResSTN, which incorporates preprocessing techniques, was designed to effectively address the low recognition rate caused by the diverse orientation distribution of individual cows. Subsequently, by constructing an open-set recognition framework that integrates three attention mechanisms, four loss functions, and four distance metric methods and exploring the impact of each component on recognition performance, this study achieves refined and optimized model configurations. Lastly, introducing moderate cropping and random occlusion strategies during the data-loading phase enhances the model's ability to recognize partially visible individuals. The method proposed in this study achieves a recognition accuracy of 94.58% in open-set scenarios for individual cows in overhead images, with an average accuracy improvement of 2.98 percentage points for cows with diverse orientation distributions, and also demonstrates an improved recognition performance for partially visible and randomly occluded individual cows. This validates the effectiveness of the proposed method in open-set recognition, showing significant potential for application in precision cattle farming management.

DifAttack: Query-Efficient Black-Box Adversarial Attack via Disentangled Feature Space

This work investigates efficient score-based black-box adversarial attacks with high Attack Success Rate (ASR) and good generalizability. We design a novel attack method based on a Disentangled Feature space, called DifAttack, which differs significantly from the existing ones operating over the entire feature space. Specifically, DifAttack firstly disentangles an image's latent feature into an adversarial feature and a visual feature, where the former dominates the adversarial capability of an image, while the latter largely determines its visual appearance. We train an autoencoder for the disentanglement by using pairs of clean images and their Adversarial Examples (AEs) generated from available surrogate models via white-box attack methods. Eventually, DifAttack iteratively optimizes the adversarial feature according to the query feedback from the victim model until a successful AE is generated, while keeping the visual feature unaltered. In addition, due to the avoidance of using surrogate models' gradient information when optimizing AEs for black-box models, our proposed DifAttack inherently possesses better attack capability in the open-set scenario, where the training dataset of the victim model is unknown. Extensive experimental results demonstrate that our method achieves significant improvements in ASR and query efficiency simultaneously, especially in the targeted attack and open-set scenarios. The code is available The code is available at https://github.com/csjunjun/DifAttack.git.

ROG_PL: Robust Open-Set Graph Learning via Region-Based Prototype Learning

Open-set graph learning is a practical task that aims to classify the known class nodes and to identify unknown class samples as unknowns. Conventional node classification methods usually perform unsatisfactorily in open-set scenarios due to the complex data they encounter, such as out-of-distribution (OOD) data and in-distribution (IND) noise. OOD data are samples that do not belong to any known classes. They are outliers if they occur in training (OOD noise), and open-set samples if they occur in testing. IND noise are training samples which are assigned incorrect labels. The existence of IND noise and OOD noise is prevalent, which usually cause the ambiguity problem, including the intra-class variety problem and the inter-class confusion problem. Thus, to explore robust open-set learning methods is necessary and difficult, and it becomes even more difficult for non-IID graph data. To this end, we propose a unified framework named ROG_PL to achieve robust open-set learning on complex noisy graph data, by introducing prototype learning. In specific, ROG_PL consists of two modules, i.e., denoising via label propagation and open-set prototype learning via regions. The first module corrects noisy labels through similarity-based label propagation and removes low-confidence samples, to solve the intra-class variety problem caused by noise. The second module learns open-set prototypes for each known class via non-overlapped regions and remains both interior and border prototypes to remedy the inter-class confusion problem. The two modules are iteratively updated under the constraints of classification loss and prototype diversity loss. To the best of our knowledge, the proposed ROG_PL is the first robust open-set node classification method for graph data with complex noise. Experimental evaluations of ROG_PL on several benchmark graph datasets demonstrate that it has good performance.

Toward Open-Set Human Object Interaction Detection

This work is oriented toward the task of open-set Human Object Interaction (HOI) detection. The challenge lies in identifying completely new, out-of-domain relationships, as opposed to in-domain ones which have seen improvements in zero-shot HOI detection. To address this challenge, we introduce a simple Disentangled HOI Detection (DHD) model for detecting novel relationships by integrating an open-set object detector with a Visual Language Model (VLM). We utilize a disentangled image-text contrastive learning metric for training and connect the bottom-up visual features to text embeddings through lightweight unary and pair-wise adapters. Our model can benefit from the open-set object detector and the VLM to detect novel action categories and combine actions with novel object categories. We further present the VG-HOI dataset, a comprehensive benchmark with over 17k HOI relationships for open-set scenarios. Experimental results show that our model can detect unknown action classes and combine unknown object classes. Furthermore, it can generalize to over 17k HOI classes while being trained on just 600 HOI classes.

A siamese-based verification system for open-set architecture attribution of synthetic images

Despite the wide variety of methods developed for synthetic image attribution, most of them can only attribute images generated by models or architectures included in the training set and do not work with unknown architectures, hindering their applicability in real-world scenarios. In this paper, we propose a verification framework that relies on a Siamese Network to address the problem of open-set attribution of synthetic images to the architecture that generated them. We consider two different settings. In the first setting, the system determines whether two images have been produced by the same generative architecture or not. In the second setting, the system verifies a claim about the architecture used to generate a synthetic image, utilizing one or multiple reference images generated by the claimed architecture. The main strength of the proposed system is its ability to operate in both closed and open-set scenarios so that the input images, either the query and reference images, can belong to the architectures considered during training or not. Experimental evaluations encompassing various generative architectures such as GANs, diffusion models, and transformers, focusing on synthetic face image generation, confirm the excellent performance of our method in both closed and open-set settings, as well as its strong generalization capabilities.

A Low-Latency Approach for RFF Identification in Open-Set Scenarios

Radio frequency fingerprint (RFF) identification represents a promising technique for lightweight device authentication. However, current research on RFF primarily focuses on the close-set recognition assumption. Moreover, the high computational complexity and excessive latency during the identification stage represent an intolerable burden for Internet of Things (IoT) devices. In this paper, we propose a deep-learning-based RFF identification framework in relation to open-set scenarios. Specifically, we leverage a simulated training scheme, in which we strategically designate certain devices as simulated unknowns. This allows us to fine-tune our extractor to better handle open-set recognition. Additionally, we construct an exemplar set that only contains representative RFF features to further reduce time consumption in the identification stage. The experiments are carried out on a hardware platform involving LoRa devices and using a USRP N210 software-defined radio receiver. The results show that the proposed framework can achieve 90.23% accuracy for rogue device detection and 93.85% accuracy for legitimate device classification. Furthermore, it is observed that using an exemplar set consisting of half the total data size can reduce the time overhead by 58% compared to using the entire dataset.

Research on the Enhancement Method of Specific Emitter Open Set Recognition

Open set recognition (OSR) aims at dealing with unknown classes that are not included in the train set. However, existing OSR methods rely on deep learning networks that perform supervised learning on known classes in the train set, resulting in poor performance when the unknown class is very similar to the known class. Considering the subtle individual differences under the same type in specific emitter identification (SEI) applications, it is difficult to distinguish between known classes and unknown classes in open set scenarios. This paper proposes a pseudo signal generation and recognition neural network (PSGRNN) to address relevant problems in this situation. PSGRNN applies complex-value convolution operations to accommodate IQ signal inputs. Its key idea is to utilize samples of known classes to generate pseudo samples of unknown classes. Then, the samples of known classes and the generated pseudo samples of unknown classes are jointly input into the neural network to construct a new classification task for training. Moreover, the center loss is improved by adding inter-class penalties to maximize the inter-class difference. This helps to learn useful information for separating known and unknown classes, resulting in clearer decision boundaries between the known and the unknown. Extensive experiments on various benchmark signal datasets indicate that the proposed method achieves more accurate and robust open set classification results, with an average accuracy improvement of 4.62%.

A Novel Distributed Fault Diagnosis Scheme Toward Open-Set Scenarios Based on Extreme Value Theory

Under closed-set scenarios (CSS), distributed modeling performs well in fault diagnosis of plant-wide industrial processes due to its flexibility and robustness. However, a more realistic scenario is often open, where unseen situations may arise unexpectedly, rendering existing methods infeasible. The advent of open-set recognition algorithms that can effectively distinguish known samples and reject unknown ones bridges this gap. Nevertheless, the poor scalability of these algorithms prevents them from being elegantly embedded in popular distributed modeling schemes, which hinders the implementation of plant-wide industrial process fault diagnosis toward open-set scenarios (OSS). In this work, we formulate a novel distributed fault diagnosis scheme toward OSS to solve this problem. Firstly, a mutual information-based local module decomposition and expansion strategy is proposed to minimize the loss of inter-module relevant information. Secondly, a novel generalized basic probability assignments generation technique based on extreme value theory is developed for modeling unknown information. It enables any classifier capable of probabilistic prediction to be applied to OSS and easily embedded in distributed modeling schemes. Finally, a conflict management scheme combining supervised and unsupervised is devised to address the vulnerability of the modified generalized combination rule to counter-intuitive results from fusing conflicting evidence. Experimental results on two plant-wide industrial process datasets demonstrate the proposed approach's feasibility and superiority.

Disentangled Representation Learning for RF Fingerprint Extraction Under Unknown Channel Statistics

Deep learning (DL) applied to a device’s radio-frequency fingerprint (RFF) has attracted significant attention in physical-layer authentication due to its extraordinary classification performance. Conventional DL-RFF techniques are trained by adopting maximum likelihood estimation (MLE). Although their discriminability has recently been extended to unknown devices in open-set scenarios, they still tend to overfit the channel statistics embedded in the training dataset. This restricts their practical applications as it is challenging to collect sufficient training data capturing the characteristics of all possible wireless channel environments. To address this challenge, we propose a DL framework of disentangled representation (DR) learning that first learns to factor the signals into a device-relevant component and a device-irrelevant component via adversarial learning. Then, it shuffles these two parts within a dataset for implicit data augmentation, which imposes a strong regularization on RFF extractor learning to avoid the possible overfitting of device-irrelevant channel statistics, without collecting additional data from unknown channels. Experiments validate that the proposed approach, referred to as DR-based RFF, outperforms conventional methods in terms of generalizability to unknown devices under unknown complicated propagation environments, e.g., dispersive multipath fading channels, even though all the training data are collected in a simple environment with dominated direct line-of-sight (LoS) propagation paths.

Multi-proxy based deep metric learning

Deep metric learning (DML) achieves excellent performance in many open-set scenarios. However, the current multi-proxy methods rely on a classification framework and the performances of these methods rely on large batch size due to the representation limitation of mini-batch for neighbor distribution of proxies. This study proposes a multi-proxy method aiming to enhance the generalization ability of DML on unseen classes by enhancing the data representation ability. As a sample may not share all fine-grained semantic information of a class, our model does not require full association between samples and proxies, and is updated only according to the associated pairs. Moreover, a mini-batch may not contain sufficient information for learning. So our model also considers the proxy-proxy relation in order to provide a global view of the data structure which improves the learning on the intra-class distance. This study also investigates how the positive and negative margins, i.e., the parameters of our model which control the required similarity for a class and different classes, affect our performance, and finds that less demanding on the negative margin avoids over-training and improves generalization ability to unseen classes. The outstanding performance of our model is confirmed by experimental results compared with the state-of-the-art DML methods in the benchmark image retrieval datasets.

Adversarial Auxiliary Weighted Subdomain Adaptation for Open-Set Deep Transfer Bridge Damage Diagnosis.

Deep learning models have been widely used in data-driven bridge structural damage diagnosis methods in recent years. However, these methods require training and test datasets to satisfy the same distribution, which is difficult to satisfy in practice. Domain adaptation transfer learning is an efficient method to solve this problem. Most of the current domain adaptation methods focus on close-set scenarios with the same classes in the source and target domains. However, in practical applications, new damage caused by long-term degradation often makes the target and source domains dissimilar in the class space. For such challenging open-set scenarios, existing domain adaptation methods will be powerless. To effectively solve the above problems, an adversarial auxiliary weighted subdomain adaptation algorithm is proposed for open-set scenarios. Adversarial learning is introduced to proposed an adversarial auxiliary weighting scheme to reflect the similarity of target samples with source classes. It effectively distinguishes unknown damage from known states. This paper further proposes a multi-channel multi-kernel weighted local maximum mean discrepancy metric (MCMK-WLMMD) to capture the fine-grained transferable information for conditional distribution alignment (sub-domain alignment). Extensive experiments on transfer tasks between three bridges verify the effectiveness of the algorithm in open-set scenarios.

Dynamic open set specific emitter identification via multi‐channel reconstructive discriminant network

AbstractSpecific emitter identification (SEI) is an emerging device authentication technology, which depends on the inherent hardware characteristics of wireless devices. By analysing the received signal, the hardware characteristics of a specific emitter can be extracted at the receiver and used for device authentication and association. Most of the existing SEI schemes focus on the identification under closed sets. In view of the explosive growth of the number of IoT devices, based on generative adversarial networks, this study proposes a method that considers the identification of unknown emitters. The reconstruction network to reconstruct the signals of the known class and fully train the feature space of the signals of the known class is designed. In the discriminator, two channels are specifically designed to perform anomaly detection for unknown signals and end‐to‐end closed‐set classification for known signals. In order to better reject unknown signals and accept known signals, Receiver operating characteristic curve and Youden index are used to determine the optimal threshold for anomaly detection. Under the given optimal threshold conditions, the identified threshold points have larger True Positive Rate. In addition, the time‐frequency feature combination vector is designed to reveal the essential characteristics of emitters. The experimental results on the real‐world datasets collected from universal software radio peripherals in short‐range communication scenario show that compared with the existing open‐set recognition methods such as C2AE, Openmax and SoftMax, the average recognition accuracy of the proposed framework improved by 0.08, 0.15, and 0.2 respectively, and the Marco‐F1 score improved by 0.05, 0.1, and 0.16, respectively, which proves the superiority of the proposed framework. In addition to identifying unknown and known Universal software radio peripherals in this article, some potential and widespread applications of the proposed framework include access user security authentication in IoT, such as Hack RF and Blue tooth devices. In addition, as a relatively general signal processing framework, the model can be used for air safety management and maritime ship identity authentication in civil aspects. In the military aspect, it can be used in electronic support and various signal reconnaissance scenarios, such as automatic signal modulation recognition in open set scenarios, individual identity determination of radar emitter and unknown working state identification of transmitters, which proves reference for subsequent research on open‐set signal recognition.

Open Set Recognition for Malware Traffic via Predictive Uncertainty

Existing machine learning-based malware traffic recognition techniques can effectively detect abnormal behaviors in the network. However, almost all of them focus on a closed-set scenario in which the data used for training and testing come from the same label space. Since sophisticated malware and advanced persistent threats are evolving, it is impossible to exhaust all attacks to train a complete recognition model under the existing technical conditions. Therefore, recognition in the real network is an open-set problem, i.e., the recognition system should identify unknown and unseen attacks at test time. In this paper, we propose an uncertainty-aware method to identify known malicious traffic accurately and handle unknown traffic effectively. This method employs predictive uncertainty in deep learning as an indicator for unknown class detection. The predictive uncertainty represents the confidence in neural network predictions. In particular, the Deep Evidence Malware Traffic Recognition (DEMTR) model is presented to provide the multi-classification probability and predictive uncertainty in open-set scenarios using evidential deep learning. We demonstrate the performance of DEMTR on the MCFP dataset. Experimental results indicate that the proposed model outperforms the baseline methods in accuracy and F1-score.

Robust tactile object recognition in open-set scenarios using Gaussian prototype learning.

Tactile object recognition is crucial for effective grasping and manipulation. Recently, it has started to attract increasing attention in robotic applications. While there are many works on tactile object recognition and they also achieved promising performances in some applications, most of them are usually limited to closed world scenarios, where the object instances to be recognition in deployment are known and the same as that of during training. Since robots usually operate in realistic open-set scenarios, they inevitably encounter unknown objects. If automation systems falsely recognize unknown objects as one of the known classes based on the pre-trained model, it can lead to potentially catastrophic consequences. It motivates us to break the closed world assumption and to study tactile object recognition in realistic open-set conditions. Although several open-set recognition methods have been proposed, they focused on visual tasks and may not be suitable for tactile recognition. It is mainly due to that these methods do not take into account the special characteristic of tactile data in their models. To this end, we develop a novel Gaussian Prototype Learning method for robust tactile object recognition. Particularly, the proposed method converts feature distributions to probabilistic representations, and exploit uncertainty for tactile recognition in open-set scenarios. Experiments on the two tactile recognition benchmarks demonstrate the effectiveness of the proposed method on open-set tasks.

Open-Set Recognition Model Based on Negative-Class Sample Feature Enhancement Learning Algorithm

In order to solve the problem that the F1-measure value and the AUROC value of some classical open-set classifier methods do not exceed 40% in high-openness scenarios, this paper proposes an algorithm combining negative-class feature enhancement learning and a Weibull distribution based on an extreme value theory representation method, which can effectively reduce the risk of open space in open-set scenarios. Firstly, the solution uses the negative-class sample feature enhancement learning algorithm to generate the negative sample point set of similar features and then compute the corresponding negative-class sample feature segmentation hypersphere. Secondly, the paired Weibull distributions from positive and negative samples are established based on the corresponding negative-class sample feature segmentation hypersphere of each class. Finally, solutions for non-linear multi-class classifications are constructed by using the Weibull and reverse Weibull distributions. Experiments on classic open datasets such as the open dataset of letter recognition, the Caltech256 open dataset, and the CIFAR100 open dataset show that when the openness is greater than 60%, the performance of the proposed method is significantly higher than other open-set support vector classifier algorithms, and the average is more than 7% higher.

Towards open-set text recognition via label-to-prototype learning

Scene text recognition is a popular research topic which is also extensively utilized in the industry. Although many methods have achieved satisfactory performance for the close-set text recognition challenges, these methods lose feasibility in open-set scenarios, where collecting data or retraining models for novel characters could yield a high cost. For example, annotating samples for foreign languages can be expensive, whereas retraining the model each time when a “novel” character is discovered from historical documents costs both time and resources. In this paper, we introduce and formulate a new open-set text recognition task which demands the capability to spot and recognize novel characters without retraining. A label-to-prototype learning framework is also proposed as a baseline for the new task. Specifically, the framework introduces a generalizable label-to-prototype mapping function to build prototypes (class centers) for both seen and unseen classes. An open-set predictor is then utilized to recognize or reject samples according to the prototypes. The implementation of rejection capability over out-of-set characters allows automatic spotting of unknown characters in the incoming data stream. Extensive experiments show that our method achieves promising performance on a variety of zero-shot, close-set, and open-set text recognition datasets.