Open-set Conditions Research Articles

Navigating Open Set Scenarios for Skeleton-Based Action Recognition

In real-world scenarios, human actions often fall outside the distribution of training data, making it crucial for models to recognize known actions and reject unknown ones. However, using pure skeleton data in such open-set conditions poses challenges due to the lack of visual background cues and the distinct sparse structure of body pose sequences. In this paper, we tackle the unexplored Open-Set Skeleton-based Action Recognition (OS-SAR) task and formalize the benchmark on three skeleton-based datasets. We assess the performance of seven established open-set approaches on our task and identify their limits and critical generalization issues when dealing with skeleton information.To address these challenges, we propose a distance-based cross-modality ensemble method that leverages the cross-modal alignment of skeleton joints, bones, and velocities to achieve superior open-set recognition performance. We refer to the key idea as CrossMax - an approach that utilizes a novel cross-modality mean max discrepancy suppression mechanism to align latent spaces during training and a cross-modality distance-based logits refinement method during testing. CrossMax outperforms existing approaches and consistently yields state-of-the-art results across all datasets and backbones. We will release the benchmark, code, and models to the community.

A Survey on Continual Semantic Segmentation: Theory, Challenge, Method and Application.

Continual learning, also known as incremental learning or life-long learning, stands at the forefront of deep learning and AI systems. It breaks through the obstacle of one-way training on close sets and enables continuous adaptive learning on open-set conditions. In the recent decade, continual learning has been explored and applied in multiple fields especially in computer vision covering classification, detection and segmentation tasks. Continual semantic segmentation (CSS), of which the dense prediction peculiarity makes it a challenging, intricate and burgeoning task. In this paper, we present a review of CSS, committing to building a comprehensive survey on problem formulations, primary challenges, universal datasets, neoteric theories and multifarious applications. Concretely, we begin by elucidating the problem definitions and primary challenges. Based on an in-depth investigation of relevant approaches, we sort out and categorize current CSS models into two main branches including data-replay and data-free sets. In each branch, the corresponding approaches are similarity-based clustered and thoroughly analyzed, following qualitative comparison and quantitative reproductions on relevant datasets. Besides, we also introduce four CSS specialities with diverse application scenarios and development tendencies. Furthermore, we develop a benchmark for CSS encompassing representative references, evaluation results and reproductions, which is available athttps://github.com/YBIO/SurveyCSS. We hope this survey can serve as a reference-worthy and stimulating contribution to the advancement of the life-long learning field, while also providing valuable perspectives for related fields.

SAR Target Recognition with Limited Training Samples in Open Set Conditions.

It is difficult to collect training samples for all types of synthetic aperture radar (SAR) targets. A realistic problem comes when unseen categories exist that are not included in training and benchmark data at the time of recognition, which is defined as open set recognition (OSR). Without the aid of side-information, generalized OSR methods used on ordinary optical images are usually not suitable for SAR images. In addition, OSR methods that require a large number of samples to participate in training are also not suitable for SAR images with the realistic situation of collection difficulty. In this regard, a task-oriented OSR method for SAR is proposed by distribution construction and relation measures to recognize targets of seen and unseen categories with limited training samples, and without any other simulation information. The method can judge category similarity to explain the unseen category. Distribution construction is realized by the graph convolutional network. The experimental results on the MSTAR dataset show that this method has a good recognition effect for the targets of both seen and unseen categories and excellent interpretation ability for unseen targets. Specifically, while recognition accuracy for seen targets remains above 95%, the recognition accuracy for unseen targets reaches 67% for the three-type classification problem, and 53% for the five-type classification problem.

Robust tactile object recognition in open-set scenarios using Gaussian prototype learning.

Tactile object recognition is crucial for effective grasping and manipulation. Recently, it has started to attract increasing attention in robotic applications. While there are many works on tactile object recognition and they also achieved promising performances in some applications, most of them are usually limited to closed world scenarios, where the object instances to be recognition in deployment are known and the same as that of during training. Since robots usually operate in realistic open-set scenarios, they inevitably encounter unknown objects. If automation systems falsely recognize unknown objects as one of the known classes based on the pre-trained model, it can lead to potentially catastrophic consequences. It motivates us to break the closed world assumption and to study tactile object recognition in realistic open-set conditions. Although several open-set recognition methods have been proposed, they focused on visual tasks and may not be suitable for tactile recognition. It is mainly due to that these methods do not take into account the special characteristic of tactile data in their models. To this end, we develop a novel Gaussian Prototype Learning method for robust tactile object recognition. Particularly, the proposed method converts feature distributions to probabilistic representations, and exploit uncertainty for tactile recognition in open-set scenarios. Experiments on the two tactile recognition benchmarks demonstrate the effectiveness of the proposed method on open-set tasks.

Open-set HRRP recognition method based on joint sparse representation

Aiming at the problem of multi-view high-resolution range profile (HRRP) target recognition under open set conditions, we proposed an open set recognition method based on joint sparse representation (JSR), which solves the problem of low recognition rate of traditional methods under open set conditions. This method is applied to the background of radar single-station observation. JSR is used to solve the reconstruction error of multi-view HRRP by the over-complete dictionary, while extreme value theory (EVT) is used to model the reconstruction error tailing of matching and non-matching categories and transform the open set identification problem into a hypothesis testing problem. During recognition, we use the reconstruction error to determine the candidate class, the scores of the matching class and non-matching class are obtained according to the confidence of tail distribution, and the weighted sum of the two is used as the category criterion to finally determine the target or candidate class outside the library. This method can effectively use the relevant information between multi-view HRRPs to improve the performance of HRRP recognition under open set conditions. The algorithm is tested with HRRP data generated from MSTAR inversion, and the results show that the performance of the proposed method is better than the mainstream open set recognition method.

Deep joint learning for language recognition

Deep learning methods for language recognition have achieved promising performance. However, most of the studies focus on frameworks for single types of acoustic features and single tasks. In this paper, we propose the deep joint learning strategies based on the Multi-Feature (MF) and Multi-Task (MT) models. First, we investigate the efficiency of integrating multiple acoustic features and explore two kinds of training constraints, one is introducing auxiliary classification constraints with adaptive weights for loss functions in feature encoder sub-networks, and the other option is introducing the Canonical Correlation Analysis (CCA) constraint to maximize the correlation of different feature representations. Correlated speech tasks, such as phoneme recognition, are applied as auxiliary tasks in order to learn related information to enhance the performance of language recognition. We analyze phoneme-aware information from different learning strategies, like joint learning on the frame-level, adversarial learning on the segment-level, and the combination mode. In addition, we present the Language-Phoneme embedding extraction structure to learn and extract language and phoneme embedding representations simultaneously. We demonstrate the effectiveness of the proposed approaches with experiments on the Oriental Language Recognition (OLR) data sets. Experimental results indicate that joint learning on the multi-feature and multi-task models extracts instinct feature representations for language identities and improves the performance, especially in complex challenges, such as cross-channel or open-set conditions.

Analyzing the Separability of SAR Classification Dataset in Open Set Conditions

In synthetic aperture radar (SAR) image classification applications, there are three categories of data, including training and benchmark data with fixed classes, as well as actual data in practical applications. A real problem comes that there exist unknown classes not included in training and benchmark data, which is defined as the open set condition. However, little work on recognizing unknown classes and analyzing the separability of SAR datasets has been developed. Motivated by this observation, this article demonstrates the difficulty of practical classification and analyzes SAR dataset separability in open set conditions. In this article, the SAR separability analyzer (SAR-SA) is proposed to model each known class as a multivariate Gaussian distribution. SAR-SA can classify the known classes and recognize the samples locating in each known distribution with low probabilities as unknown. Besides, SAR datasetwise separability index (DSI) and classwise separability index (CSI) are defined to quantify the separability in open set conditions at the dataset level and class level. DSI and CSI are effective indicators of the difficulty of SAR classification datasets. Extensive experimental results demonstrate that the DSI in open set conditions is nearly half of that in supervised conditions. Dataset with low DSI is hard to realize accurate classification in open set conditions. At the class level, even though the SAR image classes are semantically different from each other, there exists more or less overlap between the distributions of supervised known classes and unknown classes. Classes with low CSI are harder to be correctly classified and recognized.

Spatially-dependent Bayesian semantic perception under model and localization uncertainty

Semantic perception can provide autonomous robots operating under uncertainty with more efficient representation of their environment and better ability for correct loop closures than only geometric features. However, accurate inference of semantics requires measurement models that correctly capture properties of semantic detections such as viewpoint dependence, spatial correlations, and intra- and inter-class variations. Such models should also gracefully handle open-set conditions which may be encountered, keeping track of the resultant model uncertainty. We propose a method for robust visual classification of an object of interest observed from multiple views in the presence of significant localization uncertainty and classifier noise, and possible dataset shift. We use a viewpoint dependent measurement model to capture viewpoint dependence and spatial correlations in classifier scores, showing how to use it in the presence of localization uncertainty. Assuming a Bayesian classifier providing a measure of uncertainty, we show how its outputs can be fused in the context of the above model, allowing robust classification under model uncertainty when novel scenes are encountered. We present statistical evaluation of our method both in synthetic simulation, and in a 3D environment where rendered images are fed into a Deep Neural Network classifier. We compare to baseline methods in scenarios of varying difficulty showing improved robustness of our method to localization uncertainty and dataset shift. Finally, we validate our contribution w.r.t. localization uncertainty on a dataset of real-world images.

Speaker Adaptation on Hidden Markov Model using MFCC and Rasta-PLP and Comparative Study

This work compares the performance of the Mel-Frequency Cepstral Coefficient (MFCC) and Perceptual Linear Prediction (PLP) features for developing a text-dependent speaker identification system. Continuously spoken Hindi speech sentences have been used to train the HMM models using HTK toolkit for each speaker separately. The experiments have been performed using a set of 200 continuously spoken sentences with vocabulary of 20000 isolated words using a database of 100 speakers. The results show an accuracy of 92.26% recognition when PLP features have been used and accuracy of 91.18% for MFCC features. A confusion matrix has been created for all the 20 test speakers based on the recognition scores obtained for each of these speakers and their confusion with other speakers. Performance has been compared in the closed set and open set conditions of testing and as it is expected, the performance in the closed set condition is far better than the open set. We propose that if PLP features are used in place of MFCC, they may provide improvement in speaker identification accuracy by reducing the cases of false acceptance.

Prior-list intrusions in serial recall are positional.

Henson (1996) provided a number of demonstrations of error patterns in serial recall that contradict chaining models. One such error pattern concerned when participants make intrusions from prior lists: Rather than originating from random positions in the prior list, intrusions tend to be recalled in the same position as their position in the prior list, a finding which led to the endorsement of positional models of serial recall. However, all of the demonstrations of positional intrusions occurred in designs in which relatively small sets of items were repeatedly employed as stimuli. In recent years, a number of investigations have found evidence for chaining in designs in which large sets of items are employed and items are never reused across trials (open sets). We conducted 2 experiments using open sets of items to test whether a pure chaining model is a viable model for open-set conditions. Both experiments revealed that intrusions from the immediately preceding list exhibited a strong tendency to be output in the same position as their position in the prior list, suggesting the usage of positional representations in open-set designs. A chaining model that lacks positional representations provides an inadequate account of serial recall in open-set conditions.

Equivalent between the criterion and the three open set's conditions in disk-cyclicity

A vectorin a separable infinite dimensional Hilbert spaceis called disk-cyclic for bounded operatorif the orbit �α� � �:� � 0,α∈�;|∝| � 1�is dense in � . The useful tool used to discover codisk-cyclic operation is called the disk-cyclic Criterion. In this paper we will show that some equivalent conditions of the disk- cyclic Criterion are obtain.

Finite type, open set conditions and weak separation conditions

We study graph-directed iterated function systems of finite type. We show that such an IFS of finite type induces another graph-directed IFS of finite type where every strongly connected component satisfies the open set condition. We introduce the notions of topological and geometric weak separation properties, and summarize the relationship between the different separation conditions. For the induced IFS, the similarity, growth, box and Hausdorff dimensions coincide. Finally we prove that the generalized finite-type condition for graphs implies the geometric weak separation property.

The effects of stimulus set size and word frequency on verbal serial recall

Two experiments are reported which examine immediate serial recall for high-and low-frequency words. The words in each list were either repeatedly drawn from the same small pool of candidates (in the closed set conditions) or each word only ever occurred once during the experiment (in the open set conditions). The results consistently show an effect of word frequency but the effect of set size was only apparent for low-frequency words. It is argued that both frequency and set size effects reflect processes concerning the “clean-up” of degraded short-term memory traces.

A Multimedia Approach for Estimating Speech Recognition of Multilingual Clients

Spanish picture-identification materials have been created specifically for presentation in a computer-driven multimedia format. Normative performance data in conventional oral, open-set conditions and in computer-controlled pointing, closed-set conditions are being established for the Spanish Picture-Identification Task and will be forthcoming. The use of auditory/visual materials in the computer-controlled format represents an innovative use of multimedia technology that appears to be uniquely applicable for assessing the growing numbers of multilingual clients. It is reasonable to expect that word lists and corresponding illustrations will be developed soon for many common languages, perhaps stored on CD-ROM and accessed by audiologists through the touch of a button.