Absence Of Class Labels Research Articles

Unsupervised selection of acoustic indices: An experimental comparison for characterizing unlabeled audio recordings from sub-Andean forest soundscapes

We present an experimental comparison of applying six unsupervised (i.e. not relying on class labels) and almost parameterless feature selection methods for ranking acoustic indices. The study is aimed at guiding the practitioner in choosing appropriate acoustic indices when, in absence of class labels, a small but meaningful number of features to characterize soundscapes is desired. Forty acoustic indices were considered, which correspond to seventeen temporal, spectral and soundscape features, along with their basic statistics. Three publicly available soundscape datasets, registered in a sub-Andean forest, were used for the experiments; moreover, several subsets were considered according to the different times of the day. Results reveal that the Acoustic Evenness Index is the most important one in terms of representational power according to the six considered selection criteria, followed by the Acoustic Complexity Index when conditions are relaxed to examine features ranked among the top-five. Besides, the Bioacoustic Index, the Acoustic Diversity Index and the Root Mean Square Energy stand out as important features when characterizing days in their separated parts.

An apparent paradox: a classifier based on a partially classified sample may have smaller expected error rate than that if the sample were completely classified

There has been increasing interest in using semi-supervised learning to form a classifier. As is well known, the (Fisher) information in an unclassified feature with unknown class label is less (considerably less for weakly separated classes) than that of a classified feature which has known class label. Hence in the case where the absence of class labels does not depend on the data, the expected error rate of a classifier formed from the classified and unclassified features in a partially classified sample is greater than that if the sample were completely classified. We propose to treat the labels of the unclassified features as missing data and to introduce a framework for their missingness as in the pioneering work of Rubin (Biometrika 63:581–592, 1976) for missingness in incomplete data analysis. An examination of several partially classified data sets in the literature suggests that the unclassified features are not occurring at random in the feature space, but rather tend to be concentrated in regions of relatively high entropy. It suggests that the missingness of the labels of the features can be modelled by representing the conditional probability of a missing label for a feature via the logistic model with covariate depending on the entropy of the feature or an appropriate proxy for it. We consider here the case of two normal classes with a common covariance matrix where for computational convenience the square of the discriminant function is used as the covariate in the logistic model in place of the negative log entropy. Rather paradoxically, we show that the classifier so formed from the partially classified sample may have smaller expected error rate than that if the sample were completely classified.

Fast Reaction to Sudden Concept Drift in the Absence of Class Labels

A data stream can be considered as a sequence of examples that arrive continuously and are potentially unbounded, such as web page visits, sensor readings and call records. One of the serious and challenging problems that appears in a data stream is concept drift. This problem occurs when the relation between the input data and the target variable changes over time. Most existing works make an optimistic assumption that all incoming data are labelled and the class labels are available immediately. However, such an assumption is not always valid. Therefore, a lack of class labels aggravates the problem of concept drift detection. With this motivation, we propose a drift detector that reacts naturally to sudden drifts in the absence of class labels. In a novel way, the proposed detector reacts to concept drift in the absence of class labels, where the true label of an example is not necessary. Instead of monitoring the error estimates, the proposed detector monitors the diversity of a pair of classifiers, where the true label of an example is not necessary to determine whether components disagree. Using several datasets, an experimental evaluation and comparison is conducted against several existing detectors. The experiment results show that the proposed detector can detect drifts with less delay, runtime and memory usage.

Feature selection with graph mining technology

Many real world applications have problems with high dimensionality, which existing algorithms cannot overcome. A critical data preprocessing problem is feature selection, whereby its non-scalability negatively influences both the efficiency and performance of big data applications. In this research, we developed a new algorithm to reduce the dimensionality of a problem using graph-based analysis, which retains the physical meaning of the original high-dimensional feature space. Most existing feature-selection methods are based on a strong assumption that features are independent of each other. However, if the feature-selection algorithm does not take into consideration the interdependencies of the feature space, the selected data fail to correctly represent the original data. We developed a new feature-selection method to address this challenge. Our aim in this research was to examine the dependencies between features and select the optimal feature set with respect to the original data structure. Another important factor in our proposed method is that it can perform even in the absence of class labels. This is a more difficult problem that many feature-selection algorithms fail to address. In this case, they only use wrapper techniques that require a learning algorithm to select features. It is important to note that our experimental results indicates, this proposed simple ranking method performs better than other methods, independent of any particular learning algorithm used.

Distribution preserving learning for unsupervised feature selection

Abstract Selection of most relevant features from high-dimensional data is difficult especially in unsupervised learning scenario, this is because there is an absence of class labels that would guide the search for relevant features. In this work, we propose a distribution preserving feature selection (DPFS) method for unsupervised feature selection. Specifically, we select those features such that the distribution of the data can be preserved. Theoretical analysis show that our proposed DPFS method share some excellent properties of kernel method. Moreover, traditional “wrapper” and “filter” feature selection methods often involve an exhaustive search optimization, feature selection problem is treated as variable of optimization problem in our proposed method, the optimization is tractable. Extensive experimental results over various real-life data sets have demonstrated the effectiveness of the proposed algorithm.

An Unsupervised Approach for Combining Scores of Outlier Detection Techniques, Based on Similarity Measures

Outlier detection, the discovery of observations that deviates from normal behavior, has become crucial in many application domains. Numerous and diverse algorithms have been proposed to detect them. These algorithms identify outliers using precise definitions of the concept of outliers, thus their performance depends largely on the context of application. The construction of ensembles has been proposed as a solution to increase the individual capacity of each algorithm. However, the unsupervised scenario (absence of class labels) in the domains where outlier detection operates restricts the use of approaches relying on the existence of labels. In this paper, two novel unsupervised approaches using ensembles of heterogeneous types of detectors are proposed. Both approaches construct the ensemble using solely the results produced by each algorithm, identifying and giving more weight to the most suitable techniques depending on the particular dataset under examination. Through experimental evaluation in real world datasets, we demonstrate that our proposed algorithm provides a significant improvement over the base algorithms and even over existing approaches for ensemble outlier detection.

A new hybrid filter–wrapper feature selection method for clustering based on ranking

Feature selection is a common task in areas such as Pattern Recognition, Data Mining, and Machine Learning since it can help to improve prediction quality, reduce computation time and build more understandable models. Although feature selection for supervised classification has been widely studied, feature selection in the absence of class labels, namely feature selection for clustering or unsupervised feature selection, has been less addressed. Most existing unsupervised feature selection approaches suffer from the called “Bias of Criterion Values to Dimension,” which arises when feature subsets with different cardinality are evaluated by an internal evaluation clustering criterion. In this paper, we introduce a new hybrid filter–wrapper method for clustering, which combines the spectral feature selection framework using the Laplacian Score ranking and a modified Calinski–Harabasz index. The proposed method in the filter stage sorts the features according to their relevance, while in the wrapper stage, through our modified Calinski–Harabasz index that takes into account the cardinality of the feature subsets under evaluation, evaluates the features considering them as a subset rather than individually by using two well-known selection strategies. Experiments on different datasets show that the proposed method alleviates the “Bias of Criterion Values to Dimension” and, identifies and selects more relevant features than those selected by other reported hybrid filter–wrapper feature selection methods for clustering. Additionally, we also contrast our results against other filter and wrapper methods of the state-of-the-art.

Unsupervised probabilistic feature selection using ant colony optimization

Feature selection (FS) is one of the most important fields in pattern recognition, which aims to pick a subset of relevant and informative features from an original feature set. There are two kinds of FS algorithms depending on the presence of information about dataset class labels: supervised and unsupervised algorithms. Supervised approaches utilize class labels of dataset in the process of feature selection. On the other hand, unsupervised algorithms act in the absence of class labels, which makes their process more difficult. In this paper, we propose unsupervised probabilistic feature selection using ant colony optimization (UPFS). The algorithm looks for the optimal feature subset in an iterative process. In this algorithm, we utilize inter-feature information which shows the similarity between the features that leads the algorithm to decreased redundancy in the final set. In each step of the ACO algorithm, to select the next potential feature, we calculate the amount of redundancy between current feature and all those which have been selected thus far. In addition, we utilize a matrix to hold ant related pheromone which shows the rate of the co-presence of every pair of features in solutions. Afterwards, features are ranked based on a probability function extracted from the matrix; then, their m-top is returned as the final solution. We compare the performance of UPFS with 15 well-known supervised and unsupervised feature selection methods using different classifiers (support vector machine, naive Bayes, and k-nearest neighbor) on 10 well-known datasets. The experimental results show the efficiency of the proposed method compared to the previous related methods.

Unsupervised feature selection based on spectral regression from manifold learning for facial expression recognition

In this study, an unsupervised feature selection method is proposed for facial feature recognition (FER) in the absence of class labels. The contribution is the descriptive feature components selector spectral regression representative coefficient scores based on graph manifold learning from high‐dimensional feature space. The spectral regression analysis and L1‐regularised least square are then used to compute the importance of features in the original space, so that less representative features with lower coefficient scores will be removed without prior distribution assumption. To verify the performance of the authors’ method, some classifiers are used to classify facial expressions on three benchmark facial expression databases. The recognition results indicate the availability and effectiveness of the proposed method for FER.

A graph theoretic approach for unsupervised feature selection

Feature subset selection is a major problem in data mining which can help to reduce computation time, improve prediction performance, and build understandable models. Specifically, feature selection realized in the absence of class labels, namely unsupervised feature selection, is challenging and interesting. In this paper a novel graph-theoretic approach for unsupervised feature selection has been proposed. The proposed method works in three steps. In the first step, the entire feature set is represented as a weighted graph. In the second step, the features are divided into several clusters using a community detection algorithm and finally in the third step, a novel iterative search strategy based on node centrality is developed to select the final subset of features. The proposed feature selection method offers two major advantages: first, our method groups features into different clusters based on their similarities, in which the features in the same cluster are similar to each other, and to obtain the reduced redundancy set, the final subset of features is selected from different clusters. Second, the node centrality measure and term variance are used to identify the most representative and informative feature subset; hence, the optimal size of the feature subset can be automatically determined. The performance of the proposed method has been compared to those of the state-of-the-art unsupervised and supervised feature selection methods on eight benchmark classification problems. The results show that our method has produced consistently better classification accuracies.

Subspace learning for unsupervised feature selection via matrix factorization

Dimensionality reduction is an important and challenging task in machine learning and data mining. Feature selection and feature extraction are two commonly used techniques for decreasing dimensionality of the data and increasing efficiency of learning algorithms. Specifically, feature selection realized in the absence of class labels, namely unsupervised feature selection, is challenging and interesting. In this paper, we propose a new unsupervised feature selection criterion developed from the viewpoint of subspace learning, which is treated as a matrix factorization problem. The advantages of this work are four-fold. First, dwelling on the technique of matrix factorization, a unified framework is established for feature selection, feature extraction and clustering. Second, an iterative update algorithm is provided via matrix factorization, which is an efficient technique to deal with high-dimensional data. Third, an effective method for feature selection with numeric data is put forward, instead of drawing support from the discretization process. Fourth, this new criterion provides a sound foundation for embedding kernel tricks into feature selection. With this regard, an algorithm based on kernel methods is also proposed. The algorithms are compared with four state-of-the-art feature selection methods using six publicly available datasets. Experimental results demonstrate that in terms of clustering results, the proposed two algorithms come with better performance than the others for almost all datasets we experimented with here.

Efficient greedy feature selection for unsupervised learning

Reducing the dimensionality of the data has been a challenging task in data mining and machine learning applications. In these applications, the existence of irrelevant and redundant features negatively affects the efficiency and effectiveness of different learning algorithms. Feature selection is one of the dimension reduction techniques, which has been used to allow a better understanding of data and improve the performance of other learning tasks. Although the selection of relevant features has been extensively studied in supervised learning, feature selection in the absence of class labels is still a challenging task. This paper proposes a novel method for unsupervised feature selection, which efficiently selects features in a greedy manner. The paper first defines an effective criterion for unsupervised feature selection that measures the reconstruction error of the data matrix based on the selected subset of features. The paper then presents a novel algorithm for greedily minimizing the reconstruction error based on the features selected so far. The greedy algorithm is based on an efficient recursive formula for calculating the reconstruction error. Experiments on real data sets demonstrate the effectiveness of the proposed algorithm in comparison with the state-of-the-art methods for unsupervised feature selection.

Dependency-based feature selection for clustering symbolic data

Feature selection is a central problem in data analysis that have received a significant amount of attention from several disciplines, such as machine learning or pattern recognition. However, most of the research has been addressed towards supervised tasks, paying little attention to unsupervised learning. In this paper, we introduce an unsupervised feature selection method for symbolic clustering tasks. Our method is based upon the assumption that, in the absence of class labels, we can deem as irrelevant those features that exhibit low dependencies with the rest of features. Experiments with several data sets demonstrate that the proposed approach is able to detect completely irrelevant features and that, additionally, it removes other features without significantly hurting the performance of the clustering algorithm.