Open World Recognition Research Articles

Open-World Recognition in Remote Sensing: Concepts, challenges, and opportunities

Open-World Recognition in Remote Sensing: Concepts, challenges, and opportunities

Bayesian Embeddings for Few-Shot Open World Recognition.

As autonomous decision-making agents move from narrow operating environments to unstructured worlds, learning systems must move from a closed-world formulation to an open-world and few-shot setting in which agents continuously learn new classes from small amounts of information. This stands in stark contrast to modern machine learning systems that are typically designed with a known set of classes and a large number of examples for each class. In this work we extend embedding-based few-shot learning algorithms to the open-world recognition setting. We combine Bayesian non-parametric class priors with an embedding-based pre-training scheme to yield a highly flexible framework which we refer to as few-shot learning for open world recognition (FLOWR). We benchmark our framework on open-world extensions of the common MiniImageNet and TieredImageNet few-shot learning datasets. Our results show, compared to prior methods, strong classification accuracy performance and up to a 12% improvement in H-measure (a measure of novel class detection) from our non-parametric open-world few-shot learning scheme.

OWFD-UCPM: An open-world fault diagnosis scheme based on uncertainty calibration and prototype management

In real-world fault diagnosis tasks, it is never possible to enumerate all fault types beforehand since there are always unseen situations that may arise unexpectedly. Therefore, fault diagnosis is essentially an open-world recognition task. A desirable open-world fault diagnosis model must be able to perform open-set recognition (OSR) and incremental learning (IL). Classifier calibration and prototype replay are popular in two subtasks. However, existing calibration and replay strategies suffer from tampering with raw data and ignoring resource allocation, respectively. In this work, we formulate an open-world fault diagnosis scheme to address both issues. Firstly, we calibrate the uncertainty of the classifier’s prediction by assigning soft labels to the samples based on their distance from the class center. Then, Shannon entropy is employed to quantify the uncertainty as an estimate of the probability that the sample belongs to the unknown. Secondly, we adaptively manage the memory of old and new known classes according to the training accuracy. Then, based on this memory budget, a dissimilarity-based sparse subset selection algorithm is utilized to pick more diverse exemplars as prototypes for replay. Furthermore, we extend the open-set F-Measure (OSFM) to the weighted OSFM to make it applicable even when the classes are imbalanced. Experimental results on multiphase flow facility and wastewater treatment plant demonstrate the feasibility and superiority of the proposed method.

Radar Active Jamming Recognition under Open World Setting

To address the issue that conventional methods cannot recognize unknown patterns of radar jamming, this study adopts the idea of zero-shot learning (ZSL) and proposes an open world recognition method, RCAE-OWR, based on residual convolutional autoencoders, which can implement the classification of known and unknown patterns. In the supervised training phase, a residual convolutional autoencoder network structure is first constructed to extract the semantic information from a training set consisting solely of known jamming patterns. By incorporating center loss and reconstruction loss into the softmax loss function, a joint loss function is constructed to minimize the intra-class distance and maximize the inter-class distance in the jamming features. Moving to the unsupervised classification phase, a test set containing both known and unknown patterns is fed into the trained encoder, and a distance-based recognition method is utilized to classify the jamming signals. The results demonstrate that the proposed model not only achieves sufficient learning and representation of known jamming patterns but also effectively identifies and classifies unknown jamming signals. When the jamming-to-noise ratio (JNR) exceeds 10 dB, the recognition rate for seven known jamming patterns and two unknown jamming patterns is more than 92%.

Open world recognition of communication jamming signals

To improve the recognition ability of communication jamming signals, Siamese Neural Network-based Open World Recognition (SNN-OWR) is proposed. The algorithm can recognize known jamming classes, detect new (unknown) jamming classes, and unsupervised cluseter new classes. The network of SNN-OWR is trained supervised with paired input data consisting of two samples from a known dataset. On the one hand, the network is required to have the ability to distinguish whether two samples are from the same class. On the other hand, the latent distribution of known class is forced to approach their own unique Gaussian distribution, which is prepared for the subsequent open set testing. During the test, the unknown class detection process based on Gaussian probability density function threshold is designed, and an unsupervised clustering algorithm of the unknown jamming is realized by using the prior knowledge of known classes. The simulation results show that when the jamming-to-noise ratio is more than 0dB, the accuracy of SNN-OWR algorithm for known jamming classes recognition, unknown jamming detection and unsupervised clustering of unknown jamming is about 95%. This indicates that the SNN-OWR algorithm can make the effect of the recognition of unknown jamming be almost the same as that of known jamming.

Multi-modal Open World User Identification

User identification is an essential step in creating a personalised long-term interaction with robots. This requires learning the users continuously and incrementally, possibly starting from a state without any known user. In this article, we describe a multi-modal incremental Bayesian network with online learning, which is the first method that can be applied in such scenarios. Face recognition is used as the primary biometric, and it is combined with ancillary information, such as gender, age, height, and time of interaction to improve the recognition. The Multi-modal Long-term User Recognition Dataset is generated to simulate various human-robot interaction (HRI) scenarios and evaluate our approach in comparison to face recognition, soft biometrics, and a state-of-the-art open world recognition method (Extreme Value Machine). The results show that the proposed methods significantly outperform the baselines, with an increase in the identification rate up to 47.9% in open-set and closed-set scenarios, and a significant decrease in long-term recognition performance loss. The proposed models generalise well to new users, provide stability, improve over time, and decrease the bias of face recognition. The models were applied in HRI studies for user recognition, personalised rehabilitation, and customer-oriented service, which showed that they are suitable for long-term HRI in the real world.

On the Challenges of Open World Recognition Under Shifting Visual Domains

Robotic visual systems operating in the wild must act in unconstrained scenarios, under different environmental conditions while facing a variety of semantic concepts, including unknown ones. To this end, recent works tried to empower visual object recognition methods with the capability to i) detect unseen concepts and ii) extended their knowledge over time, as images of new semantic classes arrive. This setting, called Open World Recognition (OWR), has the goal to produce systems capable of breaking the semantic limits present in the initial training set. However, this training set imposes to the system not only its own semantic limits, but also environmental ones, due to its bias toward certain acquisition conditions that do not necessarily reflect the high variability of the real-world. This discrepancy between training and test distribution is called domain-shift. This letter investigates whether OWR algorithms are effective under domain-shift, presenting the first benchmark setup for assessing fairly the performances of OWR algorithms, with and without domain-shift. We then use this benchmark to conduct analyses in various scenarios, showing how existing OWR algorithms indeed suffer a severe performance degradation when train and test distributions differ. Our analysis shows that this degradation is only slightly mitigated by coupling OWR with domain generalization techniques, indicating that the mere plug-and-play of existing algorithms is not enough to recognize new and unknown categories in unseen domains. Our results clearly point toward open issues and future research directions, that need to be investigated for building robot visual systems able to function reliably under these challenging yet very real conditions.

Boosting Deep Open World Recognition by Clustering

While convolutional neural networks have brought significant advances in robot vision, their ability is often limited to closed world scenarios, where the number of semantic concepts to be recognized is determined by the available training set. Since it is practically impossible to capture all possible semantic concepts present in the real world in a single training set, we need to break the closed world assumption, equipping our robot with the capability to act in an open world. To provide such ability, a robot vision system should be able to (i) identify whether an instance does not belong to the set of known categories (i.e. open set recognition), and (ii) extend its knowledge to learn new classes over time (i.e. incremental learning). In this work, we show how we can boost the performance of deep open world recognition algorithms by means of a new loss formulation enforcing a global to local clustering of class-specific features. In particular, a first loss term, i.e. global clustering, forces the network to map samples closer to the class centroid they belong to while the second one, local clustering, shapes the representation space in such a way that samples of the same class get closer in the representation space while pushing away neighbours belonging to other classes. Moreover, we propose a strategy to learn class-specific rejection thresholds, instead of heuristically estimating a single global threshold, as in previous works. Experiments on RGB-D Object and Core50 datasets show the effectiveness of our approach.

Recent Advances in Open Set Recognition: A Survey.

In real-world recognition/classification tasks, limited by various objective factors, it is usually difficult to collect training samples to exhaust all classes when training a recognizer or classifier. A more realistic scenario is open set recognition (OSR), where incomplete knowledge of the world exists at training time, and unknown classes can be submitted to an algorithm during testing, requiring the classifiers to not only accurately classify the seen classes, but also effectively deal with unseen ones. This paper provides a comprehensive survey of existing open set recognition techniques covering various aspects ranging from related definitions, representations of models, datasets, evaluation criteria, and algorithm comparisons. Furthermore, we briefly analyze the relationships between OSR and its related tasks including zero-shot, one-shot (few-shot) recognition/learning techniques, classification with reject option, and so forth. Additionally, we also review the open world recognition which can be seen as a natural extension of OSR. Importantly, we highlight the limitations of existing approaches and point out some promising subsequent research directions in this field.

Learning and the Unknown: Surveying Steps toward Open World Recognition

As science attempts to close the gap between man and machine by building systems capable of learning, we must embrace the importance of the unknown. The ability to differentiate between known and unknown can be considered a critical element of any intelligent self-learning system. The ability to reject uncertain inputs has a very long history in machine learning, as does including a background or garbage class to account for inputs that are not of interest. This paper explains why neither of these is genuinely sufficient for handling unknown inputs – uncertain is not unknown, and unknowns need not appear to be uncertain to a learning system. The past decade has seen the formalization and development of many open set algorithms, which provably bound the risk from unknown classes. We summarize the state of the art, core ideas, and results and explain why, despite the efforts to date, the current techniques are genuinely insufficient for handling unknown inputs, especially for deep networks.