Unsupervised Clustering Techniques Research Articles

Visualizing and Interpreting Unsupervised Solar Wind Classifications

One of the goals of machine learning is to eliminate tedious and arduous repetitive work. The manual and semi-automatic classification of millions of hours of solar wind data from multiple missions can be replaced by automatic algorithms that can discover, in mountains of multi-dimensional data, the real differences in the solar wind properties. In this paper we present how unsupervised clustering techniques can be used to segregate different types of solar wind. We propose the use of advanced data reduction methods to pre-process the data, and we introduce the use of Self-Organizing Maps to visualize and interpret 14 years of ACE data. Finally, we show how these techniques can potentially be used to uncover hidden information, and how they compare with previous manual and automatic categorizations.

Acoustic feature based unsupervised approach of heart sound event detection

This paper represents an unsupervised approach to detect the positions of S1, S2 heart sound events in a Phonocardiogram (PCG) recording. Insufficiency of correctly annotated heart sound database drives us to investigate unsupervised techniques. Gammatone filter bank features are used to characterize the spectral pattern of fundamental heart sound events from noise contaminated PCG data. An unsupervised spectral clustering technique is employed for segmentation of S1/S2 and non-S1/S2 heart sound events. A Feature winning score is computed to identify the S1/S2 and non-S1/S2 frames. Finally, time based threshold is applied to detect the accurate positions of S1 and S2 heart sounds. The performance of spectral clustering is compared with other clustering methods. The proposed method offers a maximum F1-score of 98% and 92.5% for normal and abnormal PCG data respectively on 2016 PhysioNet/CinC challenge dataset. The heart sound annotation algorithm provided by PhysioNet has been used as the ground truth after hand correction.

Novel clustering schemes for full and compact polarimetric SAR data: An application for rice phenology characterization

Information on rice phenological stages from Synthetic Aperture Radar (SAR) images is of prime interest for in-season monitoring. Often, prior in-situ measurements of phenology are not available. In such situations, unsupervised clustering of SAR images might help in discriminating phenological stages of a crop throughout its growing period. Among the existing unsupervised clustering techniques using full-polarimetric (FP) SAR images, the eigenvalue-eigenvector based roll-invariant scattering-type parameter, and the scattering entropy parameter are widely used in the literature. In this study, we utilize a unique target scattering-type parameter, which jointly uses the Barakat degree of polarization and the elements of the polarimetric coherency matrix. Likewise, we also utilize an equivalent parameter proposed for compact-polarimetric (CP) SAR data. These scattering-type parameters are analogous to the Cloude-Pottier’s parameter for FP SAR data and the ellipticity parameter for CP SAR data. Besides this, we also introduce new clustering schemes for both FP and CP SAR data for segmenting diverse scattering mechanisms across the phenological stages of rice. In this study, we use the RADARSAT-2 FP and simulated CP SAR data acquired over the Indian test site of Vijayawada under the Joint Experiment for Crop Assessment and Monitoring (JECAM) initiative. The temporal analysis of the scattering-type parameters and the new clustering schemes help us to investigate detailed scattering characteristics from rice across its phenological stages.

Predicting Perceptual Quality in Internet Television Based on Unsupervised Learning

Quality of service (QoS) and quality of experience (QoE) are two major concepts for the quality evaluation of video services. QoS analyzes the technical performance of a network transmission chain (e.g., utilization or packet loss rate). On the other hand, subjective evaluation (QoE) relies on the observer’s opinion, so it cannot provide output in a form of score immediately (extensive time requirements). Although several well-known methods for objective evaluation exist (trying to adopt psychological principles of the human visual system via mathematical models), each of them has its own rating scale without an existing symmetric conversion to a standardized subjective output like MOS (mean opinion score), typically represented by a five-point rating scale. This makes it difficult for network operators to recognize when they have to apply resource reservation control mechanisms. For this reason, we propose an application (classifier) that derivates the subjective end-user quality perception based on a score of objective assessment and selected parameters of each video sequence. Our model integrates the unique benefits of unsupervised learning and clustering techniques such as overfitting avoidance or small dataset requirements. In fact, most of the published papers are based on regression models or supervised clustering. In this article, we also investigate the possibility of a graphical SOM (self-organizing map) representation called a U-matrix as a feature selection method.

Unified architecture for data-driven metadata tagging of building automation systems

This article presents a Unified Architecture (UA) for automated point tagging of Building Automation System (BAS) data, based on a combination of data-driven approaches. Advanced energy analytics applications—including fault detection and diagnostics and supervisory control—have emerged as a significant opportunity for improving the performance of our built environment. Effective application of these analytics depends on harnessing structured data from the various building control and monitoring systems, but typical BAS implementations do not employ any standardized metadata schema. While standards such as Project Haystack and Brick Schema have been developed to address this issue, the process of structuring the data, i.e., tagging the points to apply a standard metadata schema, has, to date, been a manual process. This process is typically costly, labor-intensive, and error-prone. In this work we address this gap by proposing a UA that automates the process of point tagging by leveraging the data accessible through connection to the BAS, including time-series data and the raw point names. The UA intertwines supervised classification and unsupervised clustering techniques from machine learning and leverages both their deterministic and probabilistic outputs to inform the point tagging process. Furthermore, we extend the UA to embed additional input and output data-processing modules that are designed to address the challenges associated with the real-time deployment of this automation solution. We test the UA on two datasets for real-life buildings: (i) commercial retail buildings and (ii) office buildings from the National Renewable Energy Laboratory (NREL) campus. The proposed methodology correctly applied 85–90% and 70–75% of the tags in each of these test scenarios, respectively for two significantly different building types used for testing UA's fully-functional prototype. The proposed UA, therefore, offers promising approach for automatically tagging BAS data as it reaches close to 90% accuracy. Further building upon this framework to algorithmically identify the equipment type and their relationships is an apt future research direction to pursue.

Spatiotemporal Clustering-Based Formulation Aids Multiscale Modeling

This article, written by JPT Technology Editor Chris Carpenter, contains highlights of paper SPE 195329, “A General Spatiotemporal Clustering-Based Nonlocal Formulation for Multiscale Modeling of Compartmentalized Reservoirs,” by Soheil Esmaeilzadeh, Stanford University, and Amir Salehi and Gill Hetz, Quantum Reservoir Impact, et al., prepared for the 2019 SPE Western Regional Meeting, San Jose, California, 23-26 April. The paper has not been peer reviewed. In the complete paper, a novel hybrid approach is presented in which a physics-based nonlocal modeling framework is coupled with data-driven clustering techniques to provide a fast and accurate multiscale modeling of compartmentalized reservoirs. The research adds to the literature by presenting a comprehensive work on spatiotemporal clustering for reservoir-studies applications that considers the clustering complexities, the intrinsic sparse and noisy nature of the data, and the interpretability of the outcome. Introduction History matching is the most time-consuming phase in any reservoir-simulation study. As a means of accelerating reservoir simulations, a 2018 study proposed an approach in which a reservoir is treated as a combination of multiple interconnected compartments that, under a range of uncertainty, can capture the reservoir’s response during a recovery process. In this work, the authors extend that approach to represent a reservoir in a multiscale form consisting of multiple interconnected segments. To identify segments of the reservoir, spatial, temporal, and spatiotemporal unsupervised data-mining clustering techniques are used. Then, a novel nonlocal formulation for flow in porous media is presented in which the reservoir is represented by an adjacency matrix describing the neighbor and non-neighbor connections of comprising compartments. The reservoir is divided automatically into distinct compartments in which direction-dependent multiphase-flow communication is a function of nonlocal phase potential differences. With the segmented reservoir and the uncertain parameters, a robust history-matching technique is used to reproduce the reservoir’s historical response. Finally, for the segmented and history-matched reservoir, forecasting is performed. The complete paper first describes the spatiotemporal clustering framework, followed by an overview of the compartmentalized reservoir simulation framework, in which multitank history material balance and multi tank predictive material balance are explained. Next, the history-matching approach is described. These sections are not included in the current synopsis. The complete paper concludes with results of clustering, multitank material balance, and history matching for a real-world reservoir.

Identifying naturally occurring communities of primary care providers in the English National Health Service in London

ObjectivesPrimary Care Networks (PCNs) are a new organisational hierarchy with wide-ranging responsibilities introduced in the National Health Service (NHS) Long Term Plan. The vision is that PCNs should represent ‘natural’...

Unsupervised Clustering of HRV Features Reveals Preictal Changes in Human Epilepsy.

Over a third of patients suffering from epilepsy continue to live with recurrent disabling seizures and would greatly benefit from personalized seizure forecasting. While electroencephalography (EEG) remains most popular for studying subject-specific epileptic precursors, dysfunctions of the autonomous nervous system, notably cardiac activity measured in heart rate variability (HRV), have also been associated with epileptic seizures. This work proposes an unsupervised clustering technique which aims to automatically identify preictal HRV changes in 9 patients who underwent simultaneous electrocardiography (ECG) and intracranial EEG presurgical monitoring at the University of Montreal Hospital Center. A 2-class k-means clustering combined with a quantitative preictal HRV change detection technique were adopted in a subject- and seizure-specific manner. Results indicate inter and intra-patient variability in preictal HRV changes (between 3.5 and 6.5 min before seizure onset) and a statistically significant negative correlation between the time of change in HRV state and the duration of seizures (p<0.05). The presented findings show promise for new avenues of research regarding multimodal seizure prediction and unsupervised preictal time assessment.Clinical Relevance- This study proposed an unsupervised technique for quantitatively identifying preictal HRV changes which can be eventually used to implement an ECG-based seizure forecasting algorithm.

A novel approach to optimize hierarchical vegetation mapping from hyper-temporal NDVI imagery, demonstrated at national level for Namibia

This paper presents a novel methodological approach to countrywide vegetation mapping. We used green vegetation biomass over the year as captured by coarse resolution hyper-temporal NDVI satellite-imagery, to generate vegetation mapping units at the biome, ecoregion and at the next lower hierarchical level for Namibia, excluding the Zambezi Region. Our method was based on a time series of 15 years of SPOT-VGT-MVC images each representing a specific 10-day period (dekad). The ISODATA unsupervised clustering technique was used to separately create 2–100 NDVI-cluster maps. The optimal number of temporal NDVI-clusters to represent the information on vegetation contained in the imagery was established by divergence separability statistics of all generated NDVI-clusters. The selected map consisted of legend of 81 cluster-specific temporal NDVI-profiles covering each a 15-year period of averaged NDVI data representing all pixels classified to that cluster. Then, by legend-entry using the dekad-medians of all 15 annual repeats, we produced generalized legend-entries without year-specific anomalies for each cluster. Subsequently, a hierarchical cluster analysis of these temporal NDVI-profiles was used to produce a dendrogram that generated grouping options for the 81 legend-entries. Maps with cluster-groups of 8 and 4 legend-entries resulted. The 81-cluster map and its 65 legend-entries vector version have no equivalent in published vegetation maps. The 8 cluster-group map broadly corresponds with published ecoregion level maps and the 4 cluster-group map with the published biome maps in their number of legend units. The published vegetation maps varied considerably from our NDVI-profile maps in the location of mapping unit boundaries. The agreement index between our map and published biome maps ranges from 70−93. For the ecoregion level, the agreement index is much lower, namely 51−75. Our methodological approach showed a considerably higher discretionary power for hierarchical levels and the number of vegetation mapping units than the approaches applied to previously published maps. We recommended an approach to transform our three hyper-temporal NDVI-profiles based legend-entries into more specific vegetation units. This might be accomplished by re-analysis of available, spatially-comprehensive plant species occurrence data.

Semisupervised-Learning-Based Security to Detect and Mitigate Intrusions in IoT Network

Our world is moving toward an Internet of Things (IoT) era by connecting billions of IoT. There are several security loopholes in the IoT network. Intrusion can lead to performance degradation and pose a threat to data security. Hence, there is a need for a method to detect intrusion in the IoT networks. Existing solutions use supervised-learning-based intrusion detection methods that need a huge labeled data set for better accuracy. It is not easy to source out a huge labeled data set because the size of the IoT network is huge. To overcome some of the impediments in the existing solutions, we propose a novel SDRK machine learning (ML) algorithm to detect intrusion. SDRK leverages supervised deep neural networks (DNNs) and unsupervised clustering techniques. The intrusion detection and mitigation algorithms are placed in the fog nodes that are between IoT and cloud layers. We test our proposed methodology against the data deluge (DD) attack in the testbed. The SDRK model is tested on the benchmark NSL-KDD data set. We compare the results with state-of-the-art solutions. When testing with the NSL-KDD data set, we find that SDRK detects the attacks with improved accuracy of 99.78%.

Tissue-specific and interpretable sub-segmentation of whole tumour burden on CT images by unsupervised fuzzy clustering

Background:Cancer typically exhibits genotypic and phenotypic heterogeneity, which can have prognostic significance and influence therapy response. Computed Tomography (CT)-based radiomic approaches calculate quantitative features of tumour heterogeneity at a mesoscopic level, regardless of macroscopic areas of hypo-dense (i.e., cystic/necrotic), hyper-dense (i.e., calcified), or intermediately dense (i.e., soft tissue) portions. Method:With the goal of achieving the automated sub-segmentation of these three tissue types, we present here a two-stage computational framework based on unsupervised Fuzzy C-Means Clustering (FCM) techniques. No existing approach has specifically addressed this task so far. Our tissue-specific image sub-segmentation was tested on ovarian cancer (pelvic/ovarian and omental disease) and renal cell carcinoma CT datasets using both overlap-based and distance-based metrics for evaluation. Results:On all tested sub-segmentation tasks, our two-stage segmentation approach outperformed conventional segmentation techniques: fixed multi-thresholding, the Otsu method, and automatic cluster number selection heuristics for the K-means clustering algorithm. In addition, experiments showed that the integration of the spatial information into the FCM algorithm generally achieves more accurate segmentation results, whilst the kernelised FCM versions are not beneficial. The best spatial FCM configuration achieved average Dice similarity coefficient values starting from 81.94±4.76 and 83.43±3.81 for hyper-dense and hypo-dense components, respectively, for the investigated sub-segmentation tasks. Conclusions:The proposed intelligent framework could be readily integrated into clinical research environments and provides robust tools for future radiomic biomarker validation.

Investigation of Medical Image Segmentation Using Machine Learning Based Fuzzy Reliability Function for MIoT

Nowadays the image segmentation is placed a crucial role in the medical image analysis process. The automatic image segmentation process using Medical Internet of Things (MIoT) is applied to the image for analyzing the various directions such as horizontal and vertical to identify the abnormal growth of the cells present in the human part. In the recent past less accurate, more noise, high error rate and false segmentation, lead to reduce the entire disease identification process. In this research work, automatic image segmentation process using multimodal machine learning based segmentation with fuzzy reliability function techniques used to minimize the false segmentation rate and increase the recognition accuracy. This research presenting the medical imaging of the brain and retina has been segmented for clinical experimental analysis by applying the various unsupervised clustering technique and edge detection techniques to improve efficiency of the system in MIoT environment.

Objectively Determining States of the Solar Wind Using Machine Learning

Abstract Conclusively determining the states of the solar wind will aid in tracing the origins of those states to the Sun, and in the process help to find the wind’s origin and acceleration mechanism(s). Prior studies have characterized the various states of the wind, making lists that are only partially based on objective criteria; different approaches obtain substantially different results. To uncover the unbiased states of the solar wind, we use “k-means clustering”—an unsupervised machine learning method—including constructed multipoint variables. The method allows exploration of different descriptive state variables and numbers of fundamental states (clusters). We show that the clusters reveal structures similar to those found by more ad hoc means, including coronal hole wind, interplanetary coronal mass ejections, “slow wind” (better: noncoronal hole flow), “pseudostreamers,” and stream interaction regions, but with differences that should be useful in refining our previous ideas. These results demonstrate the viability of the approach and warrant further study to understand the origin of remaining discrepancies. Complexity in k-means characterization of the wind may ultimately point to complexity at the source; studies closer to the Sun with Parker Solar Probe will help. We confirm the utility of a set of variables that can serve as a proxy for composition measurements. This proxy permits studies at high time resolution and where composition is not available. This and our recently developed unsupervised multivariate clustering technique are expected to be beneficial in the automated identification of structures and events in a variety of studies.

Correlative study and analysis for hidden patterns in text analytics unstructured data using supervised and unsupervised learning techniques

Two-third of the data generated by the internet is unstructured text in the form of e-mails, audio, video, pdf files, word documents, text documents. Extraction of these unstructured text patterns using mining techniques achieve quick access to outcomes. Textual data available at online contains different patterns and when those huge incoming unstructured data enters into the system creates a problem while organising those documents into meaningful groups. This paper discusses document classification using supervised learning by focusing on the concept-based algorithm and also deals with the hidden patterns in the documents using unsupervised clustering technique and topic-based modelling for the analysis and improvement of systematic arrangement of documents by applying k-means and LDA algorithm. Finally, this presents comparative study and importance of clustering than classification for unstructured documents.

Correlative study and analysis for hidden patterns in text analytics unstructured data using supervised and unsupervised learning techniques

Two-third of the data generated by the internet is unstructured text in the form of e-mails, audio, video, pdf files, word documents, text documents. Extraction of these unstructured text patterns using mining techniques achieve quick access to outcomes. Textual data available at online contains different patterns and when those huge incoming unstructured data enters into the system creates a problem while organising those documents into meaningful groups. This paper discusses document classification using supervised learning by focusing on the concept-based algorithm and also deals with the hidden patterns in the documents using unsupervised clustering technique and topic-based modelling for the analysis and improvement of systematic arrangement of documents by applying k-means and LDA algorithm. Finally, this presents comparative study and importance of clustering than classification for unstructured documents.

Patient Stratification Using Metabolomics to Address the Heterogeneity of Psychosis

AbstractPsychosis is a symptomatic endpoint with many causes, complicating its pathophysiological characterization and treatment. Our study applies unsupervised clustering techniques to analyze metabolomic data, acquired using 2 different tandem mass spectrometry (MS-MS) methods, from an unselected group of 120 patients with psychosis. We performed an independent analysis of each of the 2 datasets generated, by both hierarchical clustering and k-means. This led to the identification of biochemically distinct groups of patients while reducing the potential biases from any single clustering method or datatype. Using our newly developed robust clustering method, which is based on patients consistently grouped together through different methods and datasets, a total of 20 clusters were ascertained and 78 patients (or 65% of the original cohort) were placed into these robust clusters. Medication exposure was not associated with cluster formation in our study. We highlighted metabolites that constitute nodes (cluster-specific metabolites) vs hubs (metabolites in a central, shared, pathway) for psychosis. For example, 4 recurring metabolites (spermine, C0, C2, and PC.aa.C38.6) were discovered to be significant in at least 8 clusters, which were identified by at least 3 different clustering approaches. Given these metabolites were affected across multiple biochemically different patient subgroups, they are expected to be important in the overall pathophysiology of psychosis. We demonstrate how knowledge about such hubs can lead to novel antipsychotic medications. Such pathways, and thus drug targets, would not have been possible to identify without patient stratification, as they are not shared by all patients, due to the heterogeneity of psychosis.

A New Approach of Multilevel Unsupervised Clustering for Detecting Replication Level in Large Image Set

Abstract Replicate and near-replicate images are variants derived from original images that are common among the large number of images on the cloud, desktop and mobile storage areas respectively. The existence of such near-replicate an image in web search indicates the presence of redundancy and lowers the system performance. The detection of replication is challenging due to the multitude of possible variations of images. In this paper, a new system called Multilevel Clustering for Image Replication Identification (MCIRI) is presented. The proposed MCIRI system aims to identify the replication level of images over the large storage area based on multilevel unsupervised clustering techniques. The MCIRI consists of four stages: feature extraction, first level clustering, second level clustering and replication measurement. In the first stage, the MCIRI extract the three features over each individual image in the image set based on standard statistical operations such as mean, standard deviation and variance. Next stage, the MCIRI split the image feature vector of image-set into ‘K’ dissimilar groups based on K-means clustering technique. In the third stage, the MCIRI partially identifies the dissimilar cluster over each individual cluster in the cluster set of previous level clustering result. In the final stage, the MCIRI estimate the replication level of each individual cluster in the result of second level clustering based on the cluster validation measure. Experimental results show that the proposed MCIRI is better suitable to estimate the replication level in the image set.

Unsupervised Graph Clustering for community Detection in Complex Networks using Spectral Analysis

Clustering is a recent technique for a smart classification of data, where the output is a set of clusters and each cluster regroups data points having similar behaviours. Traditional clustering algorithms are those where we predefine the number of clusters as an input parameter, and we control the size of the neighbourhood, also called supervised clustering. Recently, with data evolution in term of volume and variety, supervised clustering techniques were overwhelmed and unsupervised algorithms started to appear. Spectral clustering (SC) is a graph clustering technique based on spectral analysis, and it is one of the most powerful unsupervised clustering techniques. This paper presents an application of a spectral clustering algorithm to data modelled by graphs and a comparison between the two families of SC; unnormalised SC and normalised SC. We also introduce a modification of normalised SC algorithm to make the number of clusters estimated and not given as an input parameter. Further works are needed to apply the approach to larger datasets in order to evaluate its performance against big data challenges.

Fracture identification by semi-supervised learning using conventional logs in tight sandstones of Ordos Basin, China

Fracture identification using conventional logs is a cost effective way of identifying fracture zones in reservoirs. However, there are challenging problems including complex well log responses and small amount of labelled data available from cores or image logs, making it difficult to build a prediction model with a good generalization capability. To address these problems, a semi-supervised learning method termed Laplacian support vector machine (LapSVM) is introduced in this work, which is a combination of the supervised kernel method and the unsupervised clustering method. LapSVM inherits SVM's capability of handling nonlinear problems and overcomes partially the issue of limited labelled data by using the unsupervised clustering technique with the help of abundant well log information. To examine the effectiveness of LapSVM for fracture identification in tight reservoirs, a dataset from the tight sandstones of the Ordos Basin in China is used. Both statistical and geological evaluations indicate that LapSVM outperforms other three nonlinear SVM methods tested. It has been demonstrated that LapSVM can provide an accurate and effective means for the identification of fracture zones in tight reservoirs.

Visual feature fusion and its application to support unsupervised clustering tasks

The concept of involving users in the loop of analytic workflows refers to the ability to replace heuristics with user input in machine learning and data mining tasks. For supervised tasks, user engagement generally occurs via the manipulation of training data. But for unsupervised tasks, user involvement is limited to changes in the algorithm parametrization or the input data representation, also known as features. Typically, different types of features can be extracted from raw data, and the careful selection of the extraction strategy allows users to have more control over unsupervised tasks. Nevertheless, since there is no perfect feature extractor, the combination of multiple sets of features has been explored through a process called feature fusion. Feature fusion can be readily performed when the machine learning or data mining algorithms have a cost function, such as accuracy for classification tasks. However, when such a function does not exist, user support needs to be provided, otherwise the process is impractical. In this article, we present a novel feature fusion approach that employs data samples and visualization to allow users to not only effortlessly control the combination of different feature sets but also understand the attained results. The effectiveness of our approach is confirmed by a comprehensive set of qualitative and quantitative experiments, opening up different possibilities for user-guided analytical scenarios. The ability of our approach to provide real-time feedback for feature fusion is exploited in the context of unsupervised clustering techniques, where users can perform an exploratory process to discover the best combination of features that reflects their individual perceptions about similarity.