Unknown Class Research Articles

Can Reverse Nearest Neighbors Perceive Unknowns?

A novel open set classifier is presented in this work, where the neighborhood of a test instance is determined using the principles of Reverse k-nearest neighbors (RkNN). The RkNN count of an instance can have any non-negative value less or equal to the size of the training set. While dealing with an open dataset, consisting of known and unknown classes, the zero count can provide a possible solution for detecting the unknown class. Positive RkNN count along with the nearest RkNN distance information are used to determine the known class classifications. Experiments are carried out on ten real world datasets, with various openness values on five state-of-the-art open set learners and the proposed scheme. Their performance is measured on three evaluating metrics namely accuracy, average F1 over known and unknown classes, and Known class F1. Empirical results indicate comparable to superior performance delivered by the proposed method over the state-of-the-art approaches on all but one dataset.

Distributed Cooperative Deep Transfer Learning for Industrial Image Recognition

In this paper, a novel light-weight incremental class learning algorithm for live image recognition is presented. It features a dual memory architecture and is capable of learning formerly unknown classes as well as conducting its learning across multiple instances at multiple locations without storing any images. In addition to tests on the ImageNet dataset, a prototype based upon a Raspberry Pi and a webcam is used for further evaluation: The proposed algorithm successfully allows for the performant execution of image classification tasks while learning new classes at several sites simultaneously, thereby enabling its application to various industry use cases, e.g. predictive maintenance or self-optimization.

Online Multiple Object Tracking Based on Open-Set Few-Shot Learning

How to make an online tracking model effectively adapt to newly appearing objects and object disappearance as well as appearance variations of target objects from few examples is an essential issue in multiple object tracking (MOT). Learning target appearances from few examples is a few-shot classification problem, while identifications of newly appearing objects and object disappearance has the aspect of open-set classification. In this work, we regard online MOT as open-set few-show classification to address both learning from few examples (few-shot classification) and unknown classes such as new objects (open-set classification). Specifically, we develop an embedding neural network, called VOFNet, consisting of convolutional and recurrent parts, to perform open-set few-shot classification. The convolutional part constructs a feature from an example of a target object and the recurrent part determines a representative feature of a target object from few examples. Then VOFNet is trained to provide effective features for open-set few-shot classification. Finally, we develop an online multiple object tracker based on the combination of VOFNet and the bipartite matching. The proposed tracker achieves 49.2 multiple object tracking accuracy (MOTA) with 28.9 frames per second on MOT17 dataset, which shows a significantly better trade-off between the accuracy and the speed than the existing algorithms. For example, the proposed algorithm yields about 3.17 times faster speed with 0.99 times lower accuracy than recent existing MOT algorithm [1].

Algorithm for determining the optimum number of clusters on the basis of the Kohonen neural network

The article discusses the possibilities of using artificial intelligence systems to solve clustering problems. The value of the optimality criterion for various combinations of the number of clusters and the number of neurons of the output network layer is determined. Self-organizing maps (SOM, Self Organizing Maps), developed by T. Kohonen and representing a powerful tool combining two important paradigms of data analysis - clustering and projecting, visualization of multidimensional data on a plane are considered. An example of the location of cluster nuclei after training the Kohonen neural network for different values of the number of neurons in the source layer is given. Comparing the speed of modern computers with the speed of the Kohonen neural network, with other types of neural networks, allows you to conduct a large number of network exercises in a short time, so you can use one of many methods to determine the maximum value of the function. The results of experimental studies to determine the criterion of optimality are presented in the article for different combinations of the number of clusters and the number of neurons in the original layer of the network. According to the method at the initial stage, a set of input vectors is formed, each of which includes three values. A general sequence of actions is formulated to calculate the optimal number of neurons in the output layer of the Kohonen network. The methodology presented in the article is a further development of teaching methods without a teacher. The technique proposed in the article avoids the need to specify the number of outputs of the Kohonen neural network and can be widely used both in solving data mining problems and in recognizing new unknown classes and situations in different fields.

Unknown Radar Waveform Recognition Based on Transferred Deep Learning

Radar signals are emerging constantly for urgent task because of its complex patterns and rich working modes. For some radar waveforms with known modulation methods, they can be identified by correlation between radar prior knowledge and the received signals by the reconnaissance receiver. As for the unknown radar signals, how to identify unknown radar waveforms under the condition of limited samples and low signal-to-noise ratio is a challenging problem. Aiming at the learning ability of the deep features of the image by the convolutional neural network (CNN), the reconstructed features of the time-frequency image (TFI) of the known and unknown radar waveform signals have been excavated. A decision fusion unknown radar signal identification model based on transfer deep learning and linear weight decision fusion is designed in this paper. Firstly, the CNN is trained using the known radar signals; Then, based on the transfer learning, the neurons obtained from the multiple underlying the CNN are used to represent the reconstruction feature; Finally, the performance of the single random forest classifier of the original TFI and short- time autocorrelation features images (SAFI)are fused, the identification decision of unknown signals is realized by setting linear weight to the two databases. The recognition rate of unknown new classes for small samples exceeds 80.31%, and the classification accuracy rate for known radar waveform reach more than 99.15%.

Combining Support Vector Machines with Distance-based Relative Competence Weighting for Remote Sensing Image Classification: A Case Study

Abstract A classification problem involving multi-class samples is typically divided into a set of two-class sub-problems. The pairwise probabilities produced by the binary classifiers are subsequently combined to generate a final result. However, only the binary classifiers that have been trained with the unknown real class of an unlabeled sample are relevant to the multi-class problem. A distance-based relative competence weighting (DRCW) combination mechanism can estimate the competence of the binary classifiers. In this work, we adapt the DRCW mechanism to the support vector machine (SVM) approach for the classification of remote sensing images. The application of DRCW can allow the competence of a binary classifier to be estimated from the spectral information. It is therefore possible to distinguish the relevant and irrelevant binary classifiers. The SVM+DRCW classification approach is applied to analyzing the land-use/land-cover patterns in Guangzhou, China from the remotely sensed images from Landsat-5 TM and SPOT-5. The results show that the SVM+DRVW approach can achieve higher classification accuracies compared to the conventional SVM and SVMs combined with other combination mechanisms such as weighted voting (WV) and probability estimates by pairwise coupling (PE).

A Network Traffic Classification Model Based on Metric Learning

Attacks on websites and network servers are among the most critical threats in network security. Network behavior identification is one of the most effective ways to identify malicious network intrusions. Analyzing abnormal network traffic patterns and traffic classification based on labeled network traffic data are among the most effective approaches for network behavior identification. Traditional methods for network traffic classification utilize algorithms such as Naive Bayes, Decision Tree and XGBoost. However, network traffic classification, which is required for network behavior identification, generally suffers from the problem of low accuracy even with the recently proposed deep learning models. To improve network traffic classification accuracy thus improving network intrusion detection rate, this paper proposes a new network traffic classification model, called ArcMargin, which incorporates metric learning into a convolutional neural network (CNN) to make the CNN model more discriminative. ArcMargin maps network traffic samples from the same category more closely while samples from different categories are mapped as far apart as possible. The metric learning regularization feature is called additive angular margin loss, and it is embedded in the object function of traditional CNN models. The proposed ArcMargin model is validated with three datasets and is compared with several other related algorithms. According to a set of classification indicators, the ArcMargin model is proofed to have better performances in both network traffic classification tasks and open-set tasks. Moreover, in open-set tasks, the ArcMargin model can cluster unknown data classes that do not exist in the previous training dataset.

Synthesis of unsymmetrical sulfamides and polysulfamides via SuFEx click chemistry.

As hydrogen-bond donors and acceptors, N,N′-disubstituted sulfamides have been used in a range of applications from medicinal chemistry to anion-binding catalysis. However, compared to ureas or thioureas, the utilization of this unique moiety remains marginal, in part because of a lack of general synthetic methods to access unsymmetrical sulfamides. Specifically, polysulfamides are a virtually unknown type of polymer despite their potential utility in non-covalent dynamic networks, an intense area of research in materials science. We report herein a practical and efficient process to prepare unsymmetrical sulfamides via Sulfur(vi)-Fluoride Exchange (SuFEx) click chemistry. This process was then applied to synthesize polysulfamides. Thermal analysis showed that this family of polymers possess high thermal stability and tunable glass transition temperatures. Finally, hydrolysis studies indicated that aromatic polysulfamides could be recycled back to their constituting monomers at the end of their life cycle.

Using the concept of instance typicality in instance-based learning environments involving nominal attributes

Instance-Based Learning (IBL) is a machine learning research area with focus on supervised algorithms that use the given training set as the expression of the learned concept. Usually the training instances in the set are described by vectors of attribute values and an associated class. The general ization process conducted by an instance-based algorithm happens during the classification phase, when a class should be assigned to a new instance of unknown class. Attributes that describe instances can be of different types, depending on the values they represent and, usually, can be of discrete or continuous type. A subtype of the discrete type is known as nominal. An attribute of nominal type usually represents categories and there is no order among its possible values. This paper proposes and investigates an alternative strategy for dealing with nominal attributes during the classification phase of the well-known instance-based algorithm NN (Nearest Neighbor). The proposed strategy is based on the concept of typicality of an instance, which can be taken into account as a possible tiebreaker, in situations where the new instance to be classified is equidistant from more than one nearest neighbor. Experiments using the proposed strategy and the default random strategy used by the conventional NN show that a strategy based on the concept of instance typicality can be a convenient choice to improve accuracy, when data instances have nominal attributes among the attributes that describe them.

Unknown Network Attack Detection Based on Open Set Recognition

Abstract The rapid development of the Internet of Things has led to a series of security and privacy issues. Although existing network intrusion detection technologies can identify abnormal traffic, they are mostly focused on detecting in closed sets. For a real open network environment, when an unknown attack occurs, the existing detection system cannot recognize it correctly, which will severely threaten network security. In order to solve this problem, this paper investigates how the Extreme Value Theory (EVT) is applied to unknown network attack detection system and proposes a network intrusion detection method based on open set recognition. By fitting the known classes’ post recognition activations to a Weibull distribution, we build the Open-CNN model to recalculate each activation at the penultimate level, then the pseudo-probability of unknown classes can be estimated from the activation scores of known classes, realizing the detection purpose of unknown attacks. We perform experiments on multiple datasets with different types and feature distributions. All of them obtain high detection accuracy, which proves the effectiveness and robustness of the proposed method.

Design and application data mining in academic fields

Classification is a data mining technique for a process of finding a model that explains or distinguishes a concept or class of data. The goal is to be able to estimate the class of an object whose class is unknown. Seeing the importance of education quality that must increase continuously, the study was conducted to evaluate the educational process in the department based on academic historical academic data which is used as input for which new and unknown classes of objects will be obtained. The level of competitiveness in entering a department in higher education is one of the things that becomes an assessment of department accreditation. Thus it is necessary to analyze how many students accepted into a department do not move to other department the following year. The analysis used C4.5 methods and in the process used Rapid miner software to make decision trees. The results obtained an accuracy rate of around 61.2% with confussion matrix.

Towards Real-Time Heartbeat Classification: Evaluation of Nonlinear Morphological Features and Voting Method.

Abnormal heart rhythms are one of the significant health concerns worldwide. The current state-of-the-art to recognize and classify abnormal heartbeats is manually performed by visual inspection by an expert practitioner. This is not just a tedious task; it is also error prone and, because it is performed, post-recordings may add unnecessary delay to the care. The real key to the fight to cardiac diseases is real-time detection that triggers prompt action. The biggest hurdle to real-time detection is represented by the rare occurrences of abnormal heartbeats and even more are some rare typologies that are not fully represented in signal datasets; the latter is what makes it difficult for doctors and algorithms to recognize them. This work presents an automated heartbeat classification based on nonlinear morphological features and a voting scheme suitable for rare heartbeat morphologies. Although the algorithm is designed and tested on a computer, it is intended ultimately to run on a portable i.e., field-programmable gate array (FPGA) devices. Our algorithm tested on Massachusetts Institute of Technology- Beth Israel Hospital(MIT-BIH) database as per Association for the Advancement of Medical Instrumentation(AAMI) recommendations. The simulation results show the superiority of the proposed method, especially in predicting minority groups: the fusion and unknown classes with 90.4% and 100%.

Characterizing Word Embeddings for Zero-Shot Sensor-Based Human Activity Recognition.

In this paper, we address Zero-shot learning for sensor activity recognition using word embeddings. The goal of Zero-shot learning is to estimate an unknown activity class (i.e., an activity that does not exist in a given training dataset) by learning to recognize components of activities expressed in semantic vectors. The existing zero-shot methods use mainly 2 kinds of representation as semantic vectors, attribute vector and embedding word vector. However, few zero-shot activity recognition methods based on embedding vector have been studied; especially for sensor-based activity recognition, no such studies exist, to the best of our knowledge. In this paper, we compare and thoroughly evaluate the Zero-shot method with different semantic vectors: (1) attribute vector, (2) embedding vector, and (3) expanded embedding vector and analyze their correlation to performance. Our results indicate that the performance of the three spaces is similar but the use of word embedding leads to a more efficient method, since this type of semantic vector can be generated automatically. Moreover, our suggested method achieved higher accuracy than attribute-vector methods, in cases when there exist similar information in both the given sensor data and in the semantic vector; the results of this study help select suitable classes and sensor data to build a training dataset.

Probing topological properties of a three-dimensional lattice dimer model with neural networks

Machine learning methods are being actively considered as a new tool of describing many body physics. However, so far, their capabilities has been only demonstrated in previously studied models, such as e.g. Ising model. Here, we consider a simple problem, demonstrating that neural networks can be successfully used to give new insights in statistical physics. Specifically, we consider 3D lattice dimer model, which consists of sites forming a lattice and bonds connecting the neighboring sites, in such a way that every bond can be either empty or filled with a dimer, and the total number of dimers ending at one site is fixed to be one. Dimer configurations can be viewed as equivalent if they are connected through a series of local flips, i.e. simultaneous 'rotation' of a pair of parallel neighboring dimers. It turns out that the whole set of dimer configurations on a given 3D lattice can be split into distinct topological classes, such that dimer configurations belonging to different classes are not equivalent. In this paper we identify these classes by using neural networks. More specifically, we train the neural networks to distinguish dimer configurations from two known topological classes, and after it, we test them on dimer configurations from unknown topological classes. We demonstrate that 3D lattice dimer model on a bipartite lattice can be described by an integer topological invariant ('Hopf number'), whereas lattice dimer model on a non-bipartite lattice is described by $Z_2$ invariant. Thus, we demonstrate that neural networks can be successfully used to identify new topological phases in condensed matter systems, whose existence can be later verified by other (e.g. analytical) techniques.

Gravitational alternatives to dark matter with tensor mode speed equaling the speed of light

A number of theories of gravity have been proposed as proxies for dark matter in the regime of galaxies and cosmology. The recent observations of gravitational waves (GW170817) from the merger of two neutron stars, followed by an electromagnetic counterpart (GW170817a) have placed stringent constraints on the difference of the speed of gravity to the speed of light, severely restricting the phenomenological viability of such theories. We revisit the impact of these observations on the Tensor-Vector-Scalar (TeVeS) paradigm of relativistic Modified Newtonian Dynamics (MOND) and demonstrate the existence of a previously unknown class of this paradigm where the speed of gravity always equals the speed of light. We show that this holds without altering the usual (bimetric) MOND phenomenology in galaxies.

Large-scale multi-label classification using unknown streaming images

In this paper, we investigate the large-scale multi-label image classification problem when images with unknown novel classes come in stream during the training stage. It coincides with the practical requirement that usually novel classes are detected and used to update an existing image recognition system. Most existing multi-label image classification methods cannot be directly applied in this scenario, where the training and testing stages must have the same label set. In this paper, we proposed to learn a multi-label classifier and a novel-class detector alternately to solve this problem. The multi-label classifier is learned using a convolutional neural network (CNN) from the images in the known classes. We proposed a recurrent novel-class detector which is learned in the supervised manner to detect the novel class by encoding image features with the multi-label information. In the experiment, our method is evaluated on several large-scale multi-label benchmarks including MS COCO. The results show the proposed method is comparable to most existing multi-label image classification methods, which validate its efficacy when encountering streaming images with unknown classes.

Automatic classification of sources in large astronomical catalogs

AbstractIn this paper we address two questions related to data analysis in large astronomical datasets, and we demonstrate how they can be answered making use of machine learning techniques. The first question is: how to efficiently find previously unknown or rare objects which can be expected to exist in big data samples? Using the largest existing extragalactic all-sky survey, provided by the WISE satellite, we demonstrate that, surprisingly, supervised classification methods can come to aid. The second question is: having a sufficiently large data sample, how can we look for new optimal classification schemes, possibly finding new and previously unknown classes and subclasses of sources? Based on the VIPERS cutting-edge galaxy catalog at redshift z > 0.5, we demonstrate that unsupervised classification methods can give unexpected but physically well-motivated results.

Rethinking compact abating probability modeling for open set recognition problem in Cyber-physical systems

Abstract Machine learning (ML) components, especially convolutional neural network (CNN) based components, are becoming increasingly vital in cyber-physical systems (CPS), existing great influence on the dependability of CPS. Different to the popular ML model with close set hypothesis, which limits the classes of testing samples to seen classes in training samples, ML in CPS should face the problem of real-world recognition. Open set recognition (OSR) is a more suitable model for improving the dependability of ML components, which allows the presence of unknown classes at testing time. However, the ability to reject those unknown classes is concurrently required in the model. In this paper, we find the defect in compact abating probability application to most CNN-based OSR model. In order to mitigate the defect, two different methods, OpenSoftMax and OPEB are respectively proposed. Finally, extensive experiments on caltech256 data set have been conducted to verify the effectiveness and efficiency of our proposals. Compared to the well-known CNN-based OSR method OpenMax, our first method OpenSoftMax achieves 15.55% average accuracy performance gain and 28.35% F1-score performance gain, while our second method OPEB achieves 13.17% average accuracy performance gain and 25.70% F1-score performance gain.

ASSESSING AND COMPARING THE PERFORMANCE OF ENDMEMBER EXTRACTION METHODS IN MULTIPLE CHANGE DETECTION USING HYPERSPECTRAL DATA

Abstract. Endmember extraction is a process to identify the hidden pure source signals from the mixture. Endmember finding has become increasingly important in hyperspectral data exploitation because endmembers can be used to specify unknown particular spectral classes. This paper evaluates the change detection problem in bi-temporal hyperspectral remote sensing images using the unmixing process. A complete spectral unmixing process contains estimating the number of endmembers, endmember extraction and abundance estimation. Endmember extraction is a vital step in spectral unmixing of hyperspectral images. Hyperspectral change detection by unmixing has the potential to provide subpixel information from hyperspectral images. In this study, four methods including Simplex Identification via variable Splitting and Augmented Lagrangian (SISAL), N-finder algorithm (N-FINDR), Vertex Component Analysis (VCA), and Fast algorithm for linearly Unmixing (FUN) are used to produce multiple change detection maps. This paper explores and compares the performance of these methods in multiple change detection. The empirical results reveal the superiority of the FUN method in providing multiple change map with an overall accuracy of 87% and a kappa coefficient of 0.70.

Extracting Subset of Relevant Features for Breast Cancer to Improve Accuracy of Classifier

Data mining is the essential step which identifies hidden patterns from large repositories. Medical diagnosis became a major area of current research in data mining. Machine learning technique which use statistical methods to enable machine to improve with experiences and identify hidden patterns in data like regression algorithms, clustering algorithms, classification algorithms, neural networks(ANN,CNN,DL),recommender system algorithms, Apriori algorithms, page ranking algorithms, text search and NLP(natural language processing) etc.., but due to lack of evaluation, these algorithms are unsuccessful in finding a better classifier for images to estimate accuracy of classification in medical image processing. Classification is an supervised learning which predicts the future class for an unknown object. The main purpose is to identify an unknown class by consulting with the neighbor class characteristics. Clustering can be known as unsupervised learning as it label the objects based on the scale of similar characteristics without consulting its class label. Main principle of clustering is find the distance like nearby and faraway based on their similarities and dissimilarities and groups the objects and hence can be used to identify outliers (which are far away from from the object). Feature extraction, variable selection is a method of obtaining a subset of relevant characteristics from large dataset. Too many features of a class may affect the accuracy of classifier. Therefore, feature extraction technique can be used to eliminate irrelevant attributes and increases the accuracy of classifier. In this paper we performed an induction to increase the accuracy of classifier by applying mining techniques in WEKA tool. Breast Cancer dataset is chosen from learning repository to analyze and an experimental analysis was conducted with WEKA tool using training dataset by applying naïve bayes, bayesnet, and PART, ZeroR, J48 and Random Forest techniques on the Wisconsin's dataset on Breast cancer. Finally presented the best classifier where the accuracy is more