Unknown Class Labels Research Articles

Open-world structured sequence learning via dense target encoding

Structured sequences are popularly used to describe graph data with time-evolving node features and edges. A typical real-world scenario of structured sequences is that unknown class labels continuously arrive and thus the training and testing often across different class spaces. This scenario is also referred to as the open-world learning problem on structured sequences. In this paper, we present a new Dense Open-world Structured Sequence Learning model (DOSSL for short) to learn graph streams in the open-world learning setting. To capture both structural and temporal information, DOSSL uses a GNN-based stochastic recurrent neural network for learning node representation in graph streams, then a truncated Laplacian distribution to describe the latent distribution of graph nodes, and a sampling function is used to generate node representations. Further, DOSSL learns dense target embeddings for the known classes to improve the compactness of known class distribution and reserve enough space for open-world unknown classes. The ultimate open-world classifier is optimized to detect the samples from unknown classes under the constraints of DVAE loss, label loss, class uncertainty loss, and dense target loss. Through empirical analysis conducted on real-world datasets, it has been demonstrated that the advanced technique known as DOSSL exhibits the ability to acquire precise node classifiers by harnessing the power of graph streams.

K-nearest neighbors rule combining prototype selection and local feature weighting for classification

K-Nearest Neighbors (KNN) rule is a simple yet powerful classification technique in machine learning. Nevertheless, it suffers from some drawbacks such as high memory consumption, low time efficiency, class overlapping and difficulty of setting an appropriate K value. In this study, we propose an Improved K-Nearest Neighbor rule combining Prototype Selection and Local Feature Weighting (IKNN_PSLFW) to address the above issues in one framework. Differing from conventional prototype selection, IKNN_PSLFW not only selects the representative instances as prototypes, but also preserves the information of instances that are not selected. To deal with the class overlapping problem, IKNN_PSLFW explores the feature relevance in local regions by assigning different weights to different features. For an instance with unknown class label, IKNN_PSLFW uses three classification rules corresponding to three scenarios, according to the distance between the instance and each prototype, for classification. To evaluate the performance of IKNN_PSLFW, we conduct experimental study on 20 benchmark datasets. The experimental results show that compared with the conventional KNN rule, some state-of-the-art prototype selection methods and other machine learning algorithms, the proposed IKNN_PSLFW achieves promising classification performance with high time efficiency.

Deep Bayesian Unsupervised Lifelong Learning

Lifelong Learning (LL) refers to the ability to continually learn and solve new problems with incremental available information over time while retaining previous knowledge. Much attention has been given lately to Supervised Lifelong Learning (SLL) with a stream of labelled data. In contrast, we focus on resolving challenges in Unsupervised Lifelong Learning (ULL) with streaming unlabelled data when the data distribution and the unknown class labels evolve over time. Bayesian framework is natural to incorporate past knowledge and sequentially update the belief with new data. We develop a fully Bayesian inference framework for ULL with a novel end-to-end Deep Bayesian Unsupervised Lifelong Learning (DBULL) algorithm, which can progressively discover new clusters without forgetting the past with unlabelled data while learning latent representations. To efficiently maintain past knowledge, we develop a novel knowledge preservation mechanism via sufficient statistics of the latent representation for raw data. To detect the potential new clusters on the fly, we develop an automatic cluster discovery and redundancy removal strategy in our inference inspired by Nonparametric Bayesian statistics techniques. We demonstrate the effectiveness of our approach using image and text corpora benchmark datasets in both LL and batch settings.

Semisupervised Feature Selection With Sparse Discriminative Least Squares Regression.

In big data time, selecting informative features has become an urgent need. However, due to the huge cost of obtaining enough labeled data for supervised tasks, researchers have turned their attention to semisupervised learning, which exploits both labeled and unlabeled data. In this article, we propose a sparse discriminative semisupervised feature selection (SDSSFS) method. In this method, the ϵ -dragging technique for the supervised task is extended to the semisupervised task, which is used to enlarge the distance between classes in order to obtain a discriminative solution. The flexible l2,p norm is implicitly used as regularization in the new model. Therefore, we can obtain a more sparse solution by setting smaller p . An iterative method is proposed to simultaneously learn the regression coefficients and ϵ -dragging matrix and predicting the unknown class labels. Experimental results on ten real-world datasets show the superiority of our proposed method.

Investigation on enhancing the binary classification accuracy of supervised classifiers using various transforms

The Classification is used for testing instances where the unknown class labels are assigned where the predictor features are known. This paper aims to investigate the classification performance improvement of popular supervised classification approaches using data transformation techniques. Hilbert Transform, Discrete Wavelet Transform, and Principal Component Analysis are investigated as data transformation techniques for improving the performance of four different supervised classification approaches namely K-Nearest Neighbor classifier, Random Forest Classifier, Naive Bayes Classifier, and Support Vector Machine. SONAR dataset is used in this research work and the highest Mathews Correlation Coefficient of 0.72 is attained for Random Forest Classifier.

An apparent paradox: a classifier based on a partially classified sample may have smaller expected error rate than that if the sample were completely classified

There has been increasing interest in using semi-supervised learning to form a classifier. As is well known, the (Fisher) information in an unclassified feature with unknown class label is less (considerably less for weakly separated classes) than that of a classified feature which has known class label. Hence in the case where the absence of class labels does not depend on the data, the expected error rate of a classifier formed from the classified and unclassified features in a partially classified sample is greater than that if the sample were completely classified. We propose to treat the labels of the unclassified features as missing data and to introduce a framework for their missingness as in the pioneering work of Rubin (Biometrika 63:581–592, 1976) for missingness in incomplete data analysis. An examination of several partially classified data sets in the literature suggests that the unclassified features are not occurring at random in the feature space, but rather tend to be concentrated in regions of relatively high entropy. It suggests that the missingness of the labels of the features can be modelled by representing the conditional probability of a missing label for a feature via the logistic model with covariate depending on the entropy of the feature or an appropriate proxy for it. We consider here the case of two normal classes with a common covariance matrix where for computational convenience the square of the discriminant function is used as the covariate in the logistic model in place of the negative log entropy. Rather paradoxically, we show that the classifier so formed from the partially classified sample may have smaller expected error rate than that if the sample were completely classified.

Domain Adaptation on Graphs by Learning Aligned Graph Bases

A common assumption in semi-supervised learning is that the class label function has a slow variation on the data graph, while in many problems, the label function may vary abruptly in certain graph regions, resulting in high-frequency components. Although semi-supervised learning is an ill-posed problem, it is often possible to find a source graph on which the label function has similar frequency content to the target graph where the actual classification problem is defined. In this paper, we propose a method for domain adaptation on graphs based on learning the spectrum of the label function in a source graph with many labels, and transferring the spectrum information to the target graph. When transferring the frequency content, it is not easy to share graph Fourier coefficients directly between the two independently constructed graphs, since no match exists between their Fourier bases. We solve this by learning a transformation between the Fourier bases of the two graphs that flexibly “aligns” them. The unknown class label function on the target graph is then reconstructed from the learnt spectrum while retaining consistency with the available labels. Comparative experiments suggest that the proposed algorithm often outperforms recent domain adaptation methods in various data classification applications.

A Framework for Diagnosing Kidney Disease in Diabetes Patients Using Classification Algorithms

Data mining algorithms are widely used to extract latent trends or patterns from databases. They are used to predict unknown class labels in order to help in accurate decision making process. The machine learning techniques are very useful in applications of healthcare domain. In this domain, the techniques are employed to diagnose diseases and even predicting the probabilities related to health and other aspects. Since diabetes is one of the diseases that prevail in Indians now and it causes other diseases as well, in this paper we studied the kidney disease diagnosis in diabetes patients. Kidney disease dataset is used to explore four different data mining algorithms. They are known as Naive Bayes, decision table, J48 and random forest. The dataset is used to have both training and testing sets and produce prediction details. These algorithms are evaluated by using measures like accuracy, mean absolute error (MAE) and root-mean-squared error (RMSE). The results revealed that all the machine learning algorithms are able to predict the disease. However, they differ in accuracy levels. With tenfold cross-validation, it is understood that random forest showed the highest accuracy, while Naive Bayes showed least MAE and RMSE. We built prototype application to demonstrate proof of the concept. Our experimental results revealed that the proposed framework to evaluate machine learning algorithms is useful.

Domain adaptation on graphs by learning graph topologies: theoretical analysisand an algorithm

Traditional machine learning algorithms assume that the training and test data have the same distribution, while this assumption does not necessarily hold in real applications. Domain adaptation methods take into account the deviations in data distribution. In this work, we study the problem of domain adaptation on graphs. We consider a source graph and a target graph constructed with samples drawn from data manifolds. We study the problem of estimating the unknown class labels on the target graph using the label information on the source graph and the similarity between the two graphs. We particularly focus on a setting where the target label function is learned such that its spectrum is similar to that of the source label function. We first propose a theoretical analysis of domain adaptation on graphs and present performance bounds that characterize the target classification error in terms of the properties of the graphs and the data manifolds. We show that the classification performance improves as the topologies of the graphs get more balanced, i.e. as the numbers of neighbors of different graph nodes become more proportionate, and weak edges with small weights are avoided. Our results also suggest that graph edges between too distant data samples should be avoided for good generalization performance. We then propose a graph domain adaptation algorithm inspired by our theoretical findings, which estimates the label functions while learning the source and target graph topologies at the same time. The joint graph learning and label estimation problem is formulated through an objective function relying on our performance bounds, which is minimized with an alternating optimization scheme. Experiments on synthetic and real data sets suggest that the proposed method outperforms baseline approaches.

Unsupervised PolSAR image classification based on sparse representation

ABSTRACTA novel unsupervised image classification algorithm which based on the sparse representation theory for polarimetric synthetic aperture radar (PolSAR) image is introduced in this paper. The algorithm conjunctively uses sparse representation-based classification (SRC) theory, dictionary updating method, and label smoothness constraint to update class labels. The unsupervised H//A Wishart classification method is introduced to provide the preliminary classification result, from which the initial dictionary and class labels can be extracted. An energy function is defined, and it contains two terms. The first term is based on the sparse representation theory. It reflects the cost of assigning different class labels to a pixel. The second term is label smoothness constraint. It constrains that class labels of neighbouring pixels in flat regions should be the same. By alternately minimizing the energy function, two unknown variables, dictionary and class labels are updated. Optimized class labels are the outputs to compose the final classification result. Extensive experimental results for three PolSAR datasets are analysed to verify the validity of the proposed method. Comparison with other unsupervised/supervised classification methods indicates its superiority.

Discriminative Semi-Supervised Feature Selection via Rescaled Least Squares Regression-Supplement

In this paper, we propose a Discriminative Semi-Supervised Feature Selection (DSSFS) method. In this method, a ε-dragging technique is introduced to the Rescaled Linear Square Regression in order to enlarge the distances between different classes. An iterative method is proposed to simultaneously learn the regression coefficients, ε-draggings matrix and predicting the unknown class labels. Experimental results show the superiority of DSSFS.

An integrated robust semi-supervised framework for improving cluster reliability using ensemble method for heterogeneous datasets

Data mining literature offer some clustering techniques. But when we implement even an effective clustering technique, the results are found unreliable. The efficacy of the technique come under scrutiny. Here, the proposal is about an integrated framework, which ensures the reliability of the class labels assigned to a dataset whose class labels are unknown. The model uses PSO-k-means, k-medoids, c-means and Expectation Maximization for data clustering. This model integrates their results through majority voting cluster ensemble technique to enhance reliability. The reliable outcomes serve as the training set for the classification process through Bayesian classifier, Multi Layer Perceptron, Support Vector Machine and Decision tree. The predicted class labels by majority of classifiers through bagging classifier ensemble method are included with the training set and in combination, designated as the set with known class labels. Heterogeneous datasets with unknown class labels but known number of classes, after being treated through this model would be able to find the class labels for a significant portion of the data and may be accepted with reliability. The evaluation procedure has been performed by following the Dunn's, Davies–Bouldin and Modified Goodman–Kruskal indexing techniques for internal validation and probabilistic measures such as Normalized Mutual Information, Normalized Variation of Information and Adjusted Random Index which are appropriate measures of goodness-of-fit and robustness of the final clusters. The predictive capacity of the model is also validated through probabilistic measures and external indexing techniques such as Purity Measure, Random Index and F-measure.

A new feature selection approach based on ensemble methods in semi-supervised classification

In computer aided medical system, many practical classification applications are confronted to the massive multiplication of collection and storage of data, this is especially the case in areas such as the prediction of medical test efficiency, the classification of tumors and the detection of cancers. Data with known class labels (labeled data) can be limited but unlabeled data (with unknown class labels) are more readily available. Semi-supervised learning deals with methods for exploiting the unlabeled data in addition to the labeled data to improve performance on the classification task. In this paper, we consider the problem of using a large amount of unlabeled data to improve the efficiency of feature selection in large dimensional datasets, when only a small set of labeled examples is available. We propose a new semi-supervised feature evaluation method called Optimized co-Forest for Feature Selection (OFFS) that combines ideas from co-forest and the embedded principle of selecting in Random Forest based by the permutation of out-of-bag set. We provide empirical results on several medical and biological benchmark datasets, indicating an overall significant improvement of OFFS compared to four other feature selection approaches using filter, wrapper and embedded manner in semi-supervised learning. Our method proves its ability and effectiveness to select and measure importance to improve the performance of the hypothesis learned with a small amount of labeled samples by exploiting unlabeled samples.

Mining Concept Drift from Data Streams by Unsupervised Learning

Mining is involved with knowing the unknown characteristics from the databases or gaining of Knowledge (Knowledge Discovery) from Databases to get more useful information from the database. Real time databases which are constantly changing with time, there may arise a point when traditional Data Mining techniques may not be adequate as there may be a previously unknown class label involved or new properties of data which need to be taken into consideration. Thus as time passes and new data is in the dataset, the model predicted by the data mining techniques may become less accurate. This phenomenon is known as Concept Drift. The meaning of Concept Drift is the statistical properties of the target variable, i.e. how the properties of the target variable change over the course of time. The basic idea behind the ― Mining Concept Drift from Data Stream by Unsupervised Learning‖ is to detect the Concept Drift present in the Data Stream, which is used in majority of Web-Based Applications like Fraud Detection & Span E-mail Filtering etc. The approach taken here is both for the Offline Approach & an Online Approach, which can be easily merged with the current Web-Based Applications. Some examples for Concept Drift are – In a fraud detection application the target concept may be a binary attribute FRAUDULENT with values yes or no that indicates whether a given transaction is fraudulent. Or, in a weather prediction application, there may be several target concepts such as EMPERATURE, PRESSURE, and HUMIDITY. Each of these target parameters change over time and over model should be able to accommodate these changes or the Concept. In order to overcome the problems of the Offline or Desktop based processing to detect the Concept drift (which is available), it is aimed here to move the Concept Drift Detection process to the Cloud (web) & have it for Web-Based Applications too.

Analytical Study of Some Selected Classification Algorithms in WEKA Using Real Crime Data

Data mining in the field of computer science is an answered prayer to the demand of this digital age. It is used to unravel hidden information from large volumes of data usually kept in data repositories to help improve management decision making. Classification is an essential task in data mining which is used to predict unknown class labels. It has been applied in the classification of different types of data. There are different techniques that can be applied in building a classification model. In this study the performance of these techniques such as J48 which is a type of decision tree classifier, Naïve Bayesian is a classifier that applies probability functions and ZeroR is a rule induction classifier are used. These classifiers are tested using real crime data collected from Nigeria Prisons Service. The metrics used to measure the performance of each classifier include accuracy, time, True Positive Rate (TP) Rate, False Positive (FP) Rate, Kappa Statistic, Precision and Recall. The study showed that the J48 classifier has the highest accuracy compared to other two classifiers in consideration. Choosing the right classifier for data mining task will help increase the mining accuracy.

A multinomial logistic mixed model for the prediction of categorical spatial data

In this article, the prediction problem of categorical spatial data, that is, the estimation of class occurrence probability for (target) locations with unknown class labels given observed class labels at sample (source) locations, is analyzed in the framework of generalized linear mixed models, where intermediate, latent (unobservable) spatially correlated Gaussian variables (random effects) are assumed for the observable non-Gaussian responses to account for spatial dependence information. Within such a framework, a spatial multinomial logistic mixed model is proposed specifically to model categorical spatial data. Analogous to the dual form of kriging family, the proposed model is represented as a multinomial logistic function of spatial covariances between target and source locations. The associated inference problems, such as estimation of parameters and choice of the spatial covariance function for latent variables, and the connection of the proposed model with other methods, such as the indicator variants of the kriging family (indicator kriging and indicator cokriging) and Bayesian maximum entropy, are discussed in detail. The advantages and properties of the proposed method are illustrated via synthetic and real case studies.

A geostatistical framework for categorical spatial data modeling

This paper presents a general geostatistical framework for modeling categorical spatial data, an all important information source in many scientific fields. In the proposed framework, the multi-point class occurrence probability for (target) locations with unknown class labels given observed class labels at sample (source) locations is decomposed into a weighted combination of two-point spatial interactions in two different approaches, while accounting for complex spatial interdependencies. In the first approach, two-point spatial interactions are quantified directly by transiograms, a recently proposed set of spatial continuity measures in categorical fields. The sought-after multi-point class occurrence probability is then approximated based on a general paradigm (Tau model) for integrating knowledge from interdependent diverse information sources while accounting for information redundancy between them. In the second approach, geostatistical modeling of categorical spatial data is set in the framework of generalized linear mixed models (GLMMs), where latent spatially correlated Gaussian variables are assumed for the observable non-Gaussian responses to account for spatial correlation. Instead of using Markov Chain Monte Carlo sampling to infer the assumed latent variables, an ad-hoc method is proposed to approximate the analytically intractable posterior probability of the latent variables. The advantages of the new proposed framework are analyzed and highlighted through real and synthetic cases studies involving the generation of spatial patterns via sequential indicator simulation and interpolation of categorical spatial data.