Embedding Clustering Research Articles

Detection of minor compounds in complex mineral samples from millions of spectra: A new data analysis strategy in LIBS imaging

Today, Laser-Induced Breakdown Spectroscopy (LIBS) imaging is in full change. Indeed, always more stable instrumentations are developed, which significantly increases the signal quality and naturally the analytical potential of the technique for the characterization of complex and heterogeneous samples at the micro-scale level. Obviously, other intrinsic features such as a limit of detection in the order of ppm, a high field of view and high acquisition rate make it one of the most complete chemical imaging techniques to date. It is thus possible in these conditions to acquire several million spectra from one single sample in just hours. Managing big data in LIBS imaging is the challenge ahead. In this paper, we put forward a new spectral analysis strategy, called embedded k-means clustering, for simultaneous detection of major and minor compounds and the generation of associated localization maps. A complex rock section with different phases and traces will be explored to demonstrate the value of this approach.

Patent document clustering with deep embeddings

The analysis of scientific and technical documents is crucial in the process of establishing science and technology strategies. One popular method for such analysis is for field experts to manually classify each scientific or technical document into one of several predefined technical categories. However, not only is manual classification error-prone and expensive, but it also requires extended efforts to handle frequent data updates. In contrast, machine learning and text mining techniques enable cheaper and faster operations, and can alleviate the burden on human resources. In this paper, we propose a method for extracting embedded feature vectors by applying a neural embedding approach for text features in patent documents and automatically clustering the embedding features by utilizing a deep embedding clustering method.

Deep learning parallel computing and evaluation for embedded system clustering architecture processor

In the era of intelligence, the processing of a large amount of information and various intelligent applications need to rely on embedded devices. This trend has made machine learning algorithms play an increasingly important role. High-performance embedded computing is an effective means to solve the lack of computing power of embedded devices. Aiming at the problem that the calculation amount of new intelligent embedded applications based on machine learning technology is higher, the computing power of traditional embedded systems is difficult to meet their needs, this paper studies the parallel optimization and implementation techniques of convolutional neural networks in Parallella platform. The parallel optimization strategy of convolutional neural network on the clustering architecture processor of heterogeneous multi-core system is given. Then the high-performance implementation of convolutional neural network on Parallella platform is studied, and the function of convolutional neural network system is implemented. A set of performance evaluation methods for embedded parallel processors is proposed. From the application point of S698P, the eCos operating system is selected as the platform. The single-core mode and multi-core mode are compared on the simulator GRSIM, and the parallel performance evaluation is given. Experiments have shown that the efficiency of deep learning tasks is significantly improved compared to traditional parallel methods.

A multi-label text classification method via dynamic semantic representation model and deep neural network

The increment of new words and text categories requires more accurate and robust classification methods. In this paper, we propose a novel multi-label text classification method that combines dynamic semantic representation model and deep neural network (DSRM-DNN). DSRM-DNN first utilizes word embedding model and clustering algorithm to select semantic words. Then the selected words are designated as the elements of DSRM-DNN and quantified by the weighted combination of word attributes. Finally, we construct a text classifier by combining deep belief network and back-propagation neural network. During the classification process, the low-frequency words and new words are re-expressed by the existing semantic words under sparse constraint. We evaluate the performance of DSRM-DNN on RCV1-v2, Reuters-21578, EUR-Lex, and Bookmarks. Experimental results show that our method outperforms the state-of-the-art methods.

Chronic Methadone Use Alters the CD8+ T Cell Phenotype In Vivo and Modulates Its Responsiveness Ex Vivo to Opioid Receptor and TCR Stimuli.

Endogenous opioid peptides are released at sites of injury, and their cognate G protein-coupled opioid receptors (ORs) are expressed on immune cells. Although drugs of misuse appropriate ORs, conflicting reports indicate immunostimulatory and immunosuppressive activity, in that opioid users have elevated infection risk, opioids activate innate immune cells, and opioids attenuate inflammation in murine T cell-mediated autoimmunity models. The i.v. use of drugs transmits bloodborne pathogens, particularly viruses, making the study of CD8+ T cells timely. From a cohort of nonuser controls and methadone users, we demonstrate, via t-Stochastic Neighbor Embedding and k-means cluster analysis of surface marker expression, that chronic opioid use alters human CD8+ T cell subset balance, with notable decreases in T effector memory RA+ cells. Studying global CD8+ T cell populations, there were no differences in expression of OR and several markers of functionality, demonstrating the need for finer analysis. Purified CD8+ T cells from controls respond to opioids ex vivo by increasing cytoplasmic calcium, a novel finding for OR signal transduction, likely because of cell lineage. CD8+ T cells from controls exposed to μ-OR agonists ex vivo decrease expression of activation markers CD69 and CD25, although the same markers are elevated in μ-OR-treated cells from methadone users. In contrast to control cells, T cell subsets from methadone users show decreased expression of CD69 and CD25 in response to TCR stimulus. Overall, these results indicate a direct, selective role for opioids in CD8+ T cell immune regulation via their ability to modulate cell responses through the opioid receptors and TCRs.

A robust two-gene signature for glioblastoma survival prediction.

Glioblastoma multiforme (GBM) is a highly malignant brain tumor. We explored the prognostic gene signature in 443 GBM samples by systematic bioinformatics analysis, using GSE16011 with microarray expression and corresponding clinical data from Gene Expression Omnibus as the training set. Meanwhile, patients from The Chinese Glioma Genome Atlas database (CGGA) were used as the test set and The Cancer Genome Atlas database (TCGA) as the validation set. Through Cox regression analysis, Kaplan-Meier analysis, t-distributed Stochastic Neighbor Embedding algorithm, clustering, and receiver operating characteristic analysis, a two-gene signature (GRIA2 and RYR3) associated with survival wasselected in the GSE16011 dataset. The GRIA2-RYR3 signature divided patients into two risk groups with significantly different survival in the GSE16011 dataset (median: 0.72, 95% confidence interval [CI]: 0.64-0.98, vs median: 0.98, 95% CI: 0.65-1.61 years, logrank test P < .001), the CGGA dataset (median: 0.84, 95% CI: 0.70-1.18, vs median: 1.21, 95% CI: 0.95-2.94 years, logrank test P = .0017), and the TCGA dataset (median: 1.03, 95% CI: 0.86-1.24, vs median: 1.23, 95% CI: 1.04-1.85 years, logrank test P = .0064), validating the predictive value of the signature. And the survival predictive potency of the signature was independent from clinicopathological prognostic features in multivariable Cox analysis. We found that after transfection of U87 cells with small interfering RNA, GRIA2 and RYR3 influenced the biological behaviors of proliferation, migration, and invasion of glioblastoma cells. In conclusion, the two-gene signature was a robust prognostic model to predict GBM survival.

MPC-14 BRAF V600E MUTANT OLIGODENDROGLIOMA-LIKE TUMORS WITH CHROMOSOMAL INSTABILITY IN ADOLESCENTS AND YOUNG ADULTS

We performed genome-wide methylation analysis on 136 pediatric low grade gliomas, identifying a unique cluster consisting of oligodendroglioma-like BRAF V600E mutant tumors with Recurrent gain of Chromosome 7 and loss of Chromosome 10 (OLIVER). Hierarchical clustering and t-stochastic neighbor embedding analyses cluster them with previously described pediatric-type low grade gliomas, separate from adult gliomas. OLIVERS exhibit distinct clinical behavior as temporal lobe lesions in adolescents and young adults, prolonged history of seizures and all are alive with no recurrence (follow-up 3.2 to 13.2 years). Morphogically, all showed oligodendroglioma-like features, including round nuclei with perinuclear halos, a chicken-wire pattern of branching capillaries and microcalcification. None showed astrocytic features or characteristics suggestive of high-grade tumors including necrosis or mitotic figures. All tumors harbored multiple chromosomal copy number abnormalities (more than 10 chromosomes per OLIVER), but none showed 1p/19q co-deletion or IDH1 mutation. Interestingly, one tumor showed a TERT promoter mutation. Although the series is small, OLIVER may represent a new category of IDH wild-type low grade gliomas which may be confused with molecular GBM. Further, they highlight the heterogeneity of IDH wild-type gliomas and the relatively indolent behavior of pediatric-type gliomas.

A Novel Radar HRRP Recognition Method with Accelerated T-Distributed Stochastic Neighbor Embedding and Density-Based Clustering.

High-resolution range profile (HRRP) has attracted intensive attention from radar community because it is easy to acquire and analyze. However, most of the conventional algorithms require the prior information of targets, and they cannot process a large number of samples in real time. In this paper, a novel HRRP recognition method is proposed to classify unlabeled samples automatically where the number of categories is unknown. Firstly, with the preprocessing of HRRPs, we adopt principal component analysis (PCA) for dimensionality reduction of data. Afterwards, t-distributed stochastic neighbor embedding (t-SNE) with Barnes–Hut approximation is conducted for the visualization of high-dimensional data. It proves to reduce the dimensionality, which has significantly improved the computation speed. Finally, it is exhibited that the recognition performance with density-based clustering is superior to conventional algorithms under the condition of large azimuth angle ranges and low signal-to-noise ratio (SNR).

BRAF V600E mutant oligodendroglioma-like tumors with chromosomal instability in adolescents and young adults.

We performed genome-wide methylation analysis on 136 pediatric low-grade gliomas, identifying a unique cluster consisting of three tumors with oligodendroglioma-like histology, BRAF p.V600E mutations and recurrent whole chromosome gains of 7 and loss of 10. Morphologically, all showed similar features, including a diffusely infiltrative glioma composed of round nuclei with perinuclear halos, a chicken-wire pattern of branching capillaries and microcalcification. None showed astrocytic features or characteristics suggestive of high-grade tumors including necrosis or mitotic figures. All tumors harbored multiple chromosomal copy number abnormalities (>10 chromosomes altered), but none showed 1p/19q co-deletion or IDH1 p.R132H mutation. Hierarchical clustering and t-stochastic neighbor embedding analyses from DNA methylation data cluster them more closely to previously described pediatric-type low-grade gliomas and separate from adult gliomas. These tumors exhibit distinct clinical features; they are temporal lobe lesions occurring in adolescents and young adults with a prolonged history of seizures and all are alive with no recurrence (follow-up 3.2 to 13.2years). We encountered another young adult case with quite similar pathological appearance and molecular status except for TERT promoter mutation. Although the series is small, these may represent a new category of IDH wild-type low-grade gliomas which may be confused with "molecular GBM." Further, they highlight the heterogeneity of IDH wild-type gliomas and the relatively indolent behavior of "pediatric-type" gliomas.

Application of functional deep belief network for estimating daily global solar radiation: A case study in China

Solar energy plays an essential role in environment governance and resource protection, as it is totally pollution-free and extensively accessed. An accurate knowledge of solar radiation is beneficial to the deployments of solar energy constructions, photovoltaic and thermal solar systems. In this study, a deep learning method is proposed for estimating daily global solar radiation, which is constituted by embedding clustering (EC) and functional deep belief network (DBN). Based on the curve shapes of daily solar radiation, EC divides the overall dataset into different subsets, which can be modeled separately. Knowledge from empirical radiation models is also merged as the input of functional DBN. The model can be directly applied to solar estimation in various stations due to its strong nonlinear representation. The case study in China is adopted that involves radiation data from a total of 30 stations to validate the practicability and accuracy of the proposed method. From the results, the method obtains better estimation precision with empirical knowledge, achieving 1.706 MJ/m2 of mean absolute error (MAE), 2.352 MJ/m2 of root mean square error (RMSE) and 13.71% of mean absolute percentage error (MAPE) according to the average values at the 30 stations.

Seismic facies analysis based on deep convolutional embedded clustering

Seismic facies classification takes a two-step approach: attribute extraction and seismic facies analysis by using clustering algorithms, sequentially. In general, it is clear that the choice of feature extraction is critical for successful seismic facies analysis. However, the choice of features is customarily determined by the seismic interpreters, and so the clustering result is affected by the difference in the seismic interpreters’ experience levels. It becomes challenging to extract features and identify seismic facies simultaneously. We have introduced deep convolutional embedded clustering (DCEC), which aims to simultaneously learn feature representations and cluster assignments by using deep neural networks. Our method learns mapping from the data space to a lower dimensional feature space in which it iteratively optimizes a clustering objective by building a specific loss function. We apply the method to the Modified National Institute of Standards and Technology (MNIST) data, geophysical model data, and field seismic data. In the MNIST data, the DCEC method shows better latent space of clustering results than traditional clustering methods. In the geophysical model data, the accuracy of waveform classification based on DCEC method is higher than traditional clustering methods. The results from the seismic data demonstrate that selection of input data and method has an important effect on the clustering result. In addition, our method is helpful for improving the resolution of seismic facies edges and offers the richer depositional information than the traditional clustering methods.

Semi-Supervised Deep Time-Delay Embedded Clustering for Stress Speech Analysis

Real stressed speech is affected by various aspects (individual characteristics and environment) so that the stress patterns are diverse and different on each individual. To this end, in our previous work, we performed an unsupervised clustering method that able to self-learning manner by mapping the feature representations of the stress speech and clustering tasks simultaneously, called deep time-delay embedded clustering (DTEC). However, DTEC has not confirmed yet the compatibility between the output class and informational classes. Therefore, we proposed semi-supervised time-delay embedded clustering (SDTEC) as a new framework of semi-supervised in DTEC. SDTEC incorporates the prior information of pairwise constraints in the embedding layer and simultaneously learns the feature representation and the clustering assignments. The prior information was used to guide the clustering procedure so that the points that belong to the incorrect cluster can be corrected. The effectiveness of the proposed SDTEC was evaluated by comparing it with some baseline methods in terms of the clustering error rate (CER). Moreover, to demonstrate SDTEC’s capabilities, we conducted a comprehensive ablation study. Based on experiment results, SDTEC outperformed the baseline methods and achieves state-of-the-art results in semi-supervised clustering.

Acoustic Scene Clustering Using Joint Optimization of Deep Embedding Learning and Clustering Iteration

Recent efforts have been made on acoustic scene classification in the audio signal processing community. In contrast, few studies have been conducted on acoustic scene clustering, which is a newly emerging problem. Acoustic scene clustering aims at merging the audio recordings of the same class of acoustic scene into a single cluster without using prior information and training classifiers. In this study, we propose a method for acoustic scene clustering that jointly optimizes the procedures of feature learning and clustering iteration. In the proposed method, the learned feature is a deep embedding that is extracted from a deep convolutional neural network (CNN), while the clustering algorithm is the agglomerative hierarchical clustering (AHC). We formulate a unified loss function for integrating and optimizing these two procedures. Various features and methods are compared. The experimental results demonstrate that the proposed method outperforms other unsupervised methods in terms of the normalized mutual information and the clustering accuracy. In addition, the deep embedding outperforms many state-of-the-art features.

Hands-Free User Interface for AR/VR Devices Exploiting Wearer's Facial Gestures Using Unsupervised Deep Learning.

Developing a user interface (UI) suitable for headset environments is one of the challenges in the field of augmented reality (AR) technologies. This study proposes a hands-free UI for an AR headset that exploits facial gestures of the wearer to recognize user intentions. The facial gestures of the headset wearer are detected by a custom-designed sensor that detects skin deformation based on infrared diffusion characteristics of human skin. We designed a deep neural network classifier to determine the user’s intended gestures from skin-deformation data, which are exploited as user input commands for the proposed UI system. The proposed classifier is composed of a spatiotemporal autoencoder and deep embedded clustering algorithm, trained in an unsupervised manner. The UI device was embedded in a commercial AR headset, and several experiments were performed on the online sensor data to verify operation of the device. We achieved implementation of a hands-free UI for an AR headset with average accuracy of 95.4% user-command recognition, as determined through tests by participants.

Automatic Identification of Product Usage Contexts from Online Customer Reviews

AbstractThere are three product design contexts that may significantly affect the design of a product and customer preferences towards product attributes, i.e. customer context, market context, and usage context factors. The conventional methods to gather product usage contexts may be costly and time consuming to conduct. As an alternative, this paper aims to automatically identify product usage contexts from publicly available online customer reviews. The proposed methodology consists of Preprocessing, Word Embedding, and Usage Context Clustering stages. The methodology is applied to identify usage contexts from laptop customer reviews, which results in 16 clusters of usage contexts. Furthermore, analyzing the review sentences explains the separation of “playing games” –which is more related to casual gaming, and “gaming rig” –which implies high computing power requirements. Finally, comparing customer review with manufacturer's product description may reveal a discrepancy to be investigated further by product designer, e.g. a customer suggests a laptop for basic use, although the manufacturer's description describes it for heavy use.

Clustering Continuous Wavelet Transform Characteristics of Heart Rate Variability through Unsupervised Learning.

The analysis and interpretation of physiological signals acquired non-invasively are increasingly important in Smart Health, precision medicine, and medical research. However, this analysis is hampered due to the length, complexity, and inter-subject variation of these signals, and, consequently, dimensionality reduction and clustering offer substantial benefits. Machine learning, used widely in biomedicine, is increasingly being applied to physiological time series. Among the applications of unsupervised learning, clustering is one of the most important. In this paper, an unsupervised autoen-coder architecture, deep convolutional embedded clustering, is presented as a data-driven approach to study time-frequency characteristics of heart rate variability records. An autoen-coder network is trained on continuous wavelet transforms of heart rate variability signals calculated from publicly-available annotated ECG records with a wide variety of conditions. The latent variables learned by the clustering autoencoder are low-dimensional representations of wavelet transform characteristics that can be visualized and further analyzed. The results indicate that the learned clusters correspond to beat morphologies in the electrocardiogram in many cases, but also that the reduced dimensions of the time-frequency features can potentially provide additional insights into cardiac activity and the autonomic nervous system.

Pattern detection from seating pressure distribution during wheelchair motion using deep embedded clustering.

To minimize the occurrence of ulcers in high-risk mobile individuals such as wheelchair users, it is necessary to detect all typical distribution patterns and to indentify the patterns that may be associated with pressure ulcers. However, pattern detection is difficult because the pressure distribution during motion includes a variety of patterns compared to those of static postures. Thus, the establishment of a method to detect typical patterns based on distribution patterns is important. We utilized deep embedded clustering for identification purposes. This clustering technique extracts features using auto-encoder and simultaneously optimizes data points into the clusters, which might realize good clustering performance due to the detected optimal features. We used a pressure distribution dataset that was pre-labeled by nursing experts. The dataset consisted of a total of 26944 distribution images with ten class annotations. The clustering method including traditional approaches (k-means and principal component analysis plus k-means) were compared with deep embedded clustering while the threshold to noise reduction was changing. The deep embedded clustering with 80 mmHg threshold achieved the best performance. This approach also tended to be less dependent on the threshold values.

Abstract 1635: A scalable, cloud-based, unsupervised deep learning system for identification, extraction, and summarization of potentially imperceptible patterns in whole-slide images of breast cancer tissue

Abstract Digital pathology images potentially contain novel patterns that may be perceived by modern deep learning models, but not humans. Prior unsupervised pattern recognition approaches have been used to reveal prognostically-relevant subtypes of glioblastoma (PMID: 28984190) and breast density segmentation (PMID: 26915120), and may complement supervised machine learning models trained using labeled data. In the Cancer Prevention Study II (CPS-II) cohort (PMID: 12015775), high-resolution, digitized hemotoxylin and eosin diagnostic slides are available for approximately 1,700 breast cancer cases providing an opportunity to perform unsupervised pattern recognition image analysis for epidemiologic breast cancer studies. Given the size of the dataset and complexity of the models, we constructed an end-to-end analytical pipeline, including preprocessing, feature engineering, and clustering, using cloud-based technologies that enable analysis at scale. Prior to training the unsupervised models, we faced issues converting raw images with open-source software. Specifically, OpenSlides could not open the Leica Versa SCN files due to their proprietary format while BioFormats inverted colors. To fix these issues, we altered the BioFormats library to successfully convert the files into a TIFF format. Since this issue likely affects other researchers, we are in discussions to provide the fix under a public license. TIFF formatted images were then denoised through color normalization to reduce hue variance and artifact detection to remove unwanted features such as pathologist annotations. Due to the computational complexity of analyzing the full image, images were padded with white space to ensure divisibility and broken into nine tiles of a predefined size. To further reduce computation time, uninformative tiles were filtered based on a predetermined threshold of artifact and white space composition. The remaining tiles were input to the unsupervised models. We used convolutional autoencoders, specifically a modified VGG-16 model without pretrained weights and a deep embedded clustering algorithm. These models learn representations of the images called ‘feature vectors’ and encode the images’ salient patterns. The final model was chosen based on iterative testing on a subsample of 100 images (N=21,472 tiles) and performance comparison of various VGG-inspired autoencoders. The feature vectors were clustered by K-means to summarize the information in a format suitable for statistical analyses. Our initial results show that the system captures macro-scale tissue patterns at lower magnifications (1x and 5x) and produces clusters that can be integrated into epidemiological studies of breast cancer etiology and prognosis. Citation Format: Jacob L. Evans, William Seo, Mary Macheski-Preston, Michelle Fritz, Samantha Puvanesarajah, James Hodge, Ted Gansler, Susan Gapstur, Mia M. Gaudet, Michelle Yi. A scalable, cloud-based, unsupervised deep learning system for identification, extraction, and summarization of potentially imperceptible patterns in whole-slide images of breast cancer tissue [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2019; 2019 Mar 29-Apr 3; Atlanta, GA. Philadelphia (PA): AACR; Cancer Res 2019;79(13 Suppl):Abstract nr 1635.

Static malware clustering using enhanced deep embedding method

SummaryMalware refers to any software, programs, or files that are intentionally utilised to compromise the system and cause unexpected losses to end‐users such as economical losses or privacy breaches. The rapid growth of malware makes it impossible to keep up with its progress merely via human interventions or manual analysis. One of the challenges for the human‐oriented approaches is they will cause backlog and inability to keep up with the development traces of the malware. Hence, an efficient method is needed urgently to analyse effectively and identify accurately the malware in their domain. Malware clustering has been extensively studied in the machine learning area with regards to distance functions, grouping algorithm and cluster validation. A large number of research studies have been done via behavioral analysis for clustering to achieve high performance of malware detections. However, there is a trade‐off for better detection performance between behaviorial approaches and high computational forces. Up to date, little work focuses on the deep learning representations for malware clustering. Therefore, in this paper, we propose an enhanced deep embedded clustering method to facilitate an effective and efficient malware clustering process. The new method takes advantage of linear dimensionality reduction and a customised deep neural network to learn malware representations in an orthogonal space and performs cluster assignments. Our experimental results demonstrate that the proposed clustering model outperforms the traditional K‐means method with regards to the enhanced features using various auto‐encoder, pre‐trained weight and principle component analysis (PCA).

Clustering single-cell RNA-seq data with a model-based deep learning approach

Single-cell RNA sequencing (scRNA-seq) promises to provide higher resolution of cellular differences than bulk RNA sequencing. Clustering transcriptomes profiled by scRNA-seq has been routinely conducted to reveal cell heterogeneity and diversity. However, clustering analysis of scRNA-seq data remains a statistical and computational challenge, due to the pervasive dropout events obscuring the data matrix with prevailing ‘false’ zero count observations. Here, we have developed scDeepCluster, a single-cell model-based deep embedded clustering method, which simultaneously learns feature representation and clustering via explicit modelling of scRNA-seq data generation. Based on testing extensive simulated data and real datasets from four representative single-cell sequencing platforms, scDeepCluster outperformed state-of-the-art methods under various clustering performance metrics and exhibited improved scalability, with running time increasing linearly with sample size. Its accuracy and efficiency make scDeepCluster a promising algorithm for clustering large-scale scRNA-seq data. Clustering groups of cells in single-cell RNA sequencing datasets can produce high-resolution information for complex biological questions. However, it is statistically and computationally challenging due to the low RNA capture rate, which results in a high number of false zero count observations. A deep learning approach called scDeepCluster, which efficiently combines a model for explicitly characterizing missing values with clustering, shows high performance and improved scalability with a computing time increasing linearly with sample size.