Unsupervised Classification Task Research Articles

Deep parameterizations of pairwise and triplet Markov models for unsupervised classification of sequential data

Hidden Markov models are probabilistic graphical models based on hidden and observed random variables. They are popular to address classification tasks for time series applications such as part-of-speech tagging, image segmentation, genetic sequence analysis. Direct extensions of these models, the pairwise and triplet Markov models, are considered. These models aim at relaxing the assumptions underlying the hidden Markov chain by extending the direct dependencies of the involved random variables or by considering the addition of a third latent process. While these extensions define interesting modeling capabilities that have been little explored so far, they also raise new problems such as defining the nature of their core probability distributions and their parameterization. Once the model is fixed, the unsupervised classification task (i.e. the estimation of the parameters and next of the hidden random variables) is a critical problem. These challenges are addressed, first it is shown that it is possible to embed recent deep neural networks in these models in order to exploit their full modeling power. Second, a continuous latent process in triplet Markov chains is considered. The latter aims at estimating the nature of the joint distributions of the hidden and observed random variables, in addition to their parameters. The introduction of such a continuous auxiliary latent process also offers a new way to model continuous non-stationarity in hidden Markov models. For each model that is introduced, an original unsupervised Bayesian estimation method is proposed. In particular, it takes into account the interpretability of the hidden random variables in terms of signal processing classification. Through unsupervised classification problems on synthetic and real data, it is shown that the new models outperform hidden Markov chains and their classical extensions.

ESW Edge Weights: Ensemble Stochastic Watershed Edge Weights for Hyperspectral Image Classification

Hyperspectral image (HSI) classification is a topic of active research. One of the main challenges of HSI classification is the lack of reliable labelled samples. Various semi-supervised and unsupervised classification methods are proposed to handle the low number of labelled samples. Chief among them are graph convolution networks (GCN) and their variants. These approaches exploit the graph structure for semi-supervised and unsupervised classification. While several of these methods implicitly construct edge-weights, to our knowledge, not much work has been done to estimate the edge-weights explicitly. In this article, we estimate the edge-weights explicitly and use them for the downstream classification tasks - both semi-supervised and unsupervised. The proposed edge-weights are based on two key insights - (a) Ensembles reduce the variance and (b) Classes in HSI datasets and feature similarity have only one-sided implications. That is, while same classes would have similar features, similar features do not necessarily imply the same classes. Exploiting these, we estimate the edge-weights using an aggregate of ensembles of watersheds over subsamples of features. These edge weights are evaluated for both semi-supervised and unsupervised classification tasks. The evaluation for semi-supervised tasks uses Random-Walk based approach. For the unsupervised case, we use a simple filter using a graph convolution network (GCN). In both these cases, the proposed edge weights outperform the traditional approaches to compute edge-weights - Euclidean distances and cosine similarities. Fascinatingly, with the proposed edge-weights, the simplest GCN obtained results comparable to the recent state-of-the-art.

Developing an Efficient Predictive Model Based on ML and DL Approaches to Detect Diabetes

During the last decade, some important progress on machine learning ML area have been made, especially with the apparition of a new subfield called deep learning DL and CNN networks (Convolutional Neural Networks). This new tendency is used to perform much more sophisticated algorithms allowing high performance in many disciplines such as: pattern recognition, image classification, computer vision, as well as other supervised and unsupervised classification tasks. In this work, we have developed an automatic classifier that permits to classify a number of diabetic patients based on some blood characteristics by using ML and DL approaches. Initially, we have proceeded to the classification task using many ML algorithms. Then we proposed a simple CNN model composed of many layers. Finally, we established a comparison between ML and DL algorithms. For programming task, we have used Python, Tensorflow and Keras which are the most used in the field.

Fidelity-based supervised and unsupervised learning for binary classification of quantum states

Here we develop two quantum-computational models for supervised and unsupervised classification tasks in quantum world. Presuming that the states of a set of given quantum systems (or objects) belong to one of two known classes, the objective here is to decide to which of these classes each system belongs -- without knowing its state. The supervised binary classification algorithm is based on having a training sample of quantum systems whose class memberships are already known. The unsupervised binary classification algorithm, however, uses a quantum oracle which knows the class memberships of the states of the computational basis. Both algorithms require the ability to evaluate the fidelity between states of the quantum systems with unknown states, for which here we also develop a general scheme.

Deep unsupervised multi-modal fusion network for detecting driver distraction

The risk of incurring a road traffic crash has increased year by year. Studies show that lack of attention during driving is one of the major causes of traffic accidents. In this work, in order to detect driver distraction, e.g., phone conversation, eating, texting, we introduce a deep unsupervised multi-modal fusion network, termed UMMFN. It is an end-to-end model composing of three main modules: multi-modal representation learning, multi-scale feature fusion and unsupervised driver distraction detection. The first module is to learn low-dimensional representation of multiple heterogeneous sensors using embedding subnetworks. The goal of multi-scale feature fusion is to learn both the temporal dependency for each modality and spatio dependencies from different modalities. The last module utilizes a ConvLSTM Encoder-Decoder model to perform an unsupervised classification task that is not affected by new types of driver behaviors. During the detection phase, a fine-grained detection decision can be made through calculating reconstruction error of UMMFN as a score for each captured testing data. We empirically compare the proposed approach with several state-of-the-art methods on our own multi-modal dataset for distracted driving behavior. Experimental results show that UMMFN has superior performance over the existing approaches.

P37. Artificial intelligence clustering of adult spinal deformity morphology predicts surgical characteristics, alignment, and outcomes

BACKGROUND CONTEXT AI algorithms have shown substantial promise in medical image analysis. Previous studies of ASD clusters have analyzed alignment metrics – this study sought to complement these efforts by analyzing images of anatomical landmarks. PURPOSE We hypothesized that a neural-network-based artificial intelligence (AI) algorithm would cluster preoperative lateral radiographs of into groups with distinct morphology. STUDY DESIGN/SETTING Retrospective cohort study. PATIENT SAMPLE A total of 915 patients with adult spinal deformity and preoperative lateral radiographs. OUTCOME MEASURES Schwab modifiers for SVA and PI-LL, three-column osteotomy, upper instrumented vertebrae, baseline Oswestry Disability Index, and 2-year likelihood of reaching MCID in ODI (set at -12.8). Proximal junctional kyphosis and proximal junctional failure were defined using previously published radiographic criteria. METHODS Vertebral locations for C3-L5, sacral endplate, and femoral heads were measured on lateral radiographs. Pixel locations were used to create a black-and-white overlay to the image, which was subsequently standardized for size and position using the femoral heads and sacral endplate. These images were used to train a self-organizing map (SOM). SOMs are a form of artificial neural network frequently employed in unsupervised classification tasks. RESULTS In total, 915 preoperative lateral radiographs were analyzed. A 2 × 3, toroidal SOM was trained. The mean spine shape was plotted for each cluster. Alignment, surgical characteristics, and outcomes were compared between clusters. Clusters C and D exhibited a particularly high proportion of patients with optimal (ie, modifier 0) values of PI-LL (65.0% and 68.5%) and SVA (72.8% and 53.1%). Conversely, clusters B, E, and F tended to have poor (ie, modifier ++) PI-LL (74.8%, 66.9%, and 74.6%) and SVA (75.5%, 48.6%, and 58.7%). 3-CO was most common among cluster A (26.8%), cluster B (32.6%), and cluster F (32.7%). UIV at T7-T12 was most common among cluster B (51.1%) and cluster F (60.3%). ODI CONCLUSIONS This study developed a self-organizing map that clustered preoperative lateral radiographs of ASD patients into groups with highly distinct overall spinal morphology. These clusters predicted alignment, surgical characteristics, and HRQOL. Further studies of this classification approach will expand to compare pre- and postoperative radiographs. FDA DEVICE/DRUG STATUS This abstract does not discuss or include any applicable devices or drugs.

Unsupervised Exemplar-Based Learning for Improved Document Image Classification

Many recent state-of-the-art approaches for document image classification are based on supervised feature learning that requires a large amount of labeled training data. In real-world problem of document image classification, the available amount of labeled data is limited and scarce while a large amount of unlabeled data is often available at almost no cost. In this paper, we present an approach for learning visual features for document analysis in an unsupervised way, which improves the document image classification performance without increasing the amount of annotated data. The proposed approach trains a neural network model on an auxiliary task in which every training example is associated with a different label (exemplar) and expanded to multiple images through a data augmentation technique. Thus, the learned model, which is trained in an unsupervised way, is used to boost the document classification performance. In fact, this learned model has proved to be consistently efficient in two different settings: i) as an unsupervised feature extractor to represent document images for an unsupervised classification task (i.e., clustering); and ii) in the parameters initialization of a supervised classification task trained with a small amount of annotated data. We perform experiments on the Tobacco-3482 dataset and demonstrate the capability of our approach to improve i) the unsupervised classification accuracy up to 2.4%; and ii) the supervised classification accuracy by 1.5% without any extra data or by 5% when using 3000 additional not annotated samples.

Transient Variability in SOI-Based LIF Neuron and Impact on Unsupervised Learning

Variability is an integral part of biology. A biological neural network performs efficiently despite variability and sometimes its performance is facilitated by the variability. Hence, the study of variability on its electronic analog is essential for constructing biomimetic neural networks. We have recently demonstrated a compact leaky integrate and fire (LIF) neuron on PD-silicon on insulator (SOI) MOSFET. In this paper, we have studied impact ionization (II)-induced variability both device-to-device (D2D) and cycle-to-cycle (C2C) in the SOI neuron. The C2C variability is attributed to the fluctuation in the II-generated charge storage and it is enhanced by at least $2.5\times $ as compared to the no-II case. The D2D variability, on the other hand, is related to the II-induced sharp subthreshold slope (~40 mV/decade), which enhanced the variability by $\sim 20\times $ compared to the no-II case. The impact of the enhanced variability in SOI neurons on an unsupervised classification task was evaluated by simulating a spiking neural network (SNN) with both analog and binary synapses. For analog synapse-based SNN, the C2C variability improved the performance by ~5% relative to ideal LIF neurons. However, the D2D variability, as well as combined D2D and C2C variability, degrades learning by −~10%. For binary synapses, we observe that performance drastically degrades for ideal LIF neurons as the synaptic weight initialization becomes nonrandom. However, neurons with the experimentally demonstrated variability (C2C and D2D) mitigate this challenge. Therefore, this enables binary synapses to perform at par with analog synapses, which allows for deterministic weight initialization. This makes RNG circuits for random weight initialization redundant.

Unsupervised feature extraction by low-rank and sparsity preserving embedding

Manifold based feature extraction has been proved to be an effective technique in dealing with the unsupervised classification tasks. However, most of the existing works cannot guarantee the global optimum of the learned projection, and they are sensitive to different noises. In addition, many methods cannot catch the discriminative information as much as possible since they only exploit the local structure of data while ignoring the global structure. To address the above problems, this paper proposes a novel graph based feature extraction method named low-rank and sparsity preserving embedding (LRSPE) for unsupervised learning. LRSPE attempts to simultaneously learn the graph and projection in a framework so that the global optimal projection can be obtained. Moreover, LRSPE exploits both global and local information of data for projection learning by imposing the low-rank and sparse constraints on the graph, which promotes the method to obtain a better performance. Importantly, LRSPE is more robust to noise by imposing the l2,1 sparsity norm on the reconstruction errors. Experimental results on both clean and noisy datasets prove that the proposed method can significantly improve classification accuracy and it is robust to different noises in comparison with the state-of-the-art methods.

Integrating fitness predator optimizer with multi-objective PSO for dynamic partitional clustering

Clustering is an unsupervised classification task in data mining, which partitions data set into certain number of clusters. Though many metaheuristics have been suggested to optimize clustering results, most of them are marred by the three key issues. First, clustering performance is devised on a single validity measure; hence, it produces a single best solution biased toward the criterion; it is unacceptable as clustering has multiple conflicting criteria. Second, they find difficulty in avoiding local optima owing to lack of balancing in exploration and exploitation in the search space. Third, they require number of clusters a priori which is difficult to decide in the real-world problems. To deal with the first issue, we follow Pareto-based approach to obtain diverse trade-off solutions by optimizing conceptually contradicting validity measures sum of squared error and connectedness. The well-known swarm intelligence-based metaheuristic particle swarm optimization (PSO) has a severe drawback that it converges prematurely owing to single directional information sharing mechanism among particles in the swarm. Therefore, for the second issue, we introduce fitness predator optimizer (FPO) to enhance diversity in the PSO in multi-objective optimization scenario; it is named as FPO-MOPSO. To address the third issue, we use dynamic clustering method to decide an optimum number of clusters (k) in the range of 2 to \(\sqrt{n}\), where n is number of objects in the data set. We use eight real-world and two artificial data sets for comparison of the FPO-MOPSO results with three well-known competitive methods MOPSO, MABC, and NSGA-II using four quality measures convergence, diversity, coverage, and ONVG. The obtained results show superiority of the FPO-MOPSO over its competitors. Two external validity indices of clustering results F-measure and Rand index also establish the above finding. As the decision maker picks only a single solution from the set of trade-off solutions, we employ an additional measure silhouette index to select a final single solution from the archive and compare it with known number of clusters in the data sets.

Unsupervised Classification of Planning Instances

In this paper we introduce a novel approach for unsupervised classification of planning instances based on the recent formalism of planning programs. Our approach is inspired by structured prediction in machine learning, which aims at predicting structured information about a given input rather than a scalar value. In our case, each input is an unlabelled classical planning instance, and the associated structured information is the planning program that solves the instance. We describe a method that takes as input a set of planning instances and outputs a set of planning programs, classifying each instance according to the program that solves it. Our results show that automated planning can be successfully used to solve structured unsupervised classification tasks, and invites further exploration of the connection between automated planning and structured prediction.

Use of line based symmetry for developing cluster validity indices

From a dataset automatically identifying possible count of clusters is an important task of unsupervised classification. To address this issue, in the current paper, we have focused on the symmetry property of any cluster. Point and line symmetry are two important attributes of data partitions. Here we have proposed line symmetry versions of eight well-known validity indices: XB, PBM, FCM, PS, FS, K, SV, and DB indices to make them capable of identifying the accurate count of partitions from data sets containing clusters having line symmetric property. The global optimality of two of these newly developed indices is established mathematically. Eight artificially generated data sets of varying dimensions containing clusters of different convexities and shapes and three real-life data sets are used for the purpose of experiment. Initially, to obtain different partitions an existing genetic clustering technique which uses line symmetry property (GALS clustering) is applied on data sets varying the count of clusters. queryPlease check and confirm the edit in the following sentence: We have also provided a comparative study of our proposed line-symmetry-based cluster validity indices with their point-symmetry-based versions and original versions based on Euclidean distance. We have also provided a comparative study of our proposed line-symmetry-based cluster validity indices with their point-symmetry-based versions and original versions based on Euclidean distance. From the experimental results it is revealed that most of the line-symmetry-distance-based cluster validity indices perform better than their point symmetry and Euclidean-distance-based versions.

A new approach for spectral feature extraction and for unsupervised classification of hyperspectral data based on the Gaussian mixture model

This paper considers the task of unsupervised classification of hyperspectral data within a Gaussian mixture modeling framework. Different from the traditional clustering techniques that face serious conceptual problems in complex situations, the Gaussian mixture modeling approach provides a means of solving both simple and complex classification tasks as well as a way to substantiate results. Despite its theoretical advantages, in practice, the approach is rarely used for remote sensing because of objective difficulties in its implementation, especially for hyperspectral data. To show how these difficulties can be successfully overcome we present here a method for Gaussian mixture‐based unsupervised classification of hyperspectral data. The method contains a new approach for spectral feature extraction that reduces the data dimensionality while quantitatively guaranteeing the safety of the relevant information. The unsupervised learning procedure we propose is aimed at finding the number of classes that delivers a local maximum to the user's confidence in subsequent labeling the pixels. The class parameters are determined via the Expectation‐Maximization algorithm, starting from the initial values selected systematically by the learning procedure. Decisions on class membership for the pixels are proposed to be made on the basis of the lower confidence bounds of the posterior probability estimates. This allows users to provide a probabilistic guarantee of the results for each separate pixel. The potential capabilities of the method are illustrated by its application to the GER 63 channel scanner hyperspectral data of the Naan area, central Israel. A comparison of the method with one of the traditional techniques we have performed confirms the advantages of the presented method, even though just a simplified version of the method has been employed.

Unsupervised pattern classification by neural networks

A neural network is applied to the unsupervised pattern classification approach. Given a set of data consisting of unlabeled samples from several classes, the task of unsupervised classification is to label every sample in the same class by the same symbol such that the data set is divided into several clusters. We consider the hypothesis that the data set is drawn from a finite mixture of Gaussian distributions. The network architecture is a two-layer feedforward type: the units of the first layer are Gaussians and each correspond to one component of the mixture. The output layer provides the probability density estimation of the mixture. The weighted competitive learning is used to estimate the mean vectors and the non-diagonal covariance matrices of the Gaussian units. The number of Gaussian units in the hidden layer is optimized by informational criteria. Some of the results are reported, and the performance of this approach is evaluated.