Deep Clustering Research Articles

Audio–Visual Deep Clustering for Speech Separation

Speech separation aims to separate individual voices from an audio mixture of multiple simultaneous talkers. Audio-only approaches show unsatisfactory performance when the speakers are of the same gender or share similar voice characteristics. This is due to challenges on learning appropriate feature representations for separating voices in single frames and streaming voices across time. Visual signals of speech (e.g., lip movements), if available, can be leveraged to learn better feature representations for separation. In this paper, we propose a novel audio–visual deep clustering model (AVDC) to integrate visual information into the process of learning better feature representations (embeddings) for Time–Frequency (T–F) bin clustering. It employs a two-stage audio–visual fusion strategy where speaker-wise audio–visual T–F embeddings are first computed after the first-stage fusion to model the audio–visual correspondence for each speaker. In the second-stage fusion, audio–visual embeddings of all speakers and audio embeddings calculated by deep clustering from the audio mixture are concatenated to form the final T–F embedding for clustering. Through a series of experiments, the proposed AVDC model is shown to outperform the audio-only deep clustering and utterance-level permutation invariant training baselines and three other state-of-the-art audio–visual approaches. Further analyses show that the AVDC model learns a better T–F embedding for alleviating the source permutation problem across frames. Other experiments show that the AVDC model is able to generalize across different numbers of speakers between training and testing and shows some robustness when visual information is partially missing.

A Perspective Interrogation of Students’ Epistemological Beliefs And Anticipations of Studying Physics as an Introductory Course in Computer Science Engineering

Physics is ruminated to be the backbone of every Engineering stream. However, students pursuing computer science Engineering have an altogether different perception of the subject in terms of nature of knowledge and its future implications in their field of study. They are very apprehensive in stipulated learning of the subject as there exists a common belief that algorithm and program designing requires no prior knowledge of Physics or Physics laboratory and it is merely a superfluous subject laden upon them in their introductory year of Engineering. In this present manuscript, efforts are being made to explore their epistemological beliefs and expectations about the subject. In order to explore the anticipations and attitudes in learning Physics by these students, the Maryland Physics Expectation Survey (MPEX Survey) and the Engineering Physics Survey (EP Survey) has been tested for both their rationality and dependability. The EP Survey is specifically designed by the one author (N.H.) to determine the perceptions, beliefs and expectations of Computer Science Engineering students from the subject Engineering Physics. Parametric and non-parametric tests were applied to the clusters. A deep cluster wise analysis of all the items in both the surveys clearly pointed out to the fact that the students who liked Physics at school level have positive responses which are similar to that of experts and are highly motivated in learning the subject. However, the percentage of these students is quite low. Conversely, students who disliked Physics at school level carry the same perception while pursuing Engineering as well and they find no rationale in understanding the importance of this subject. Thus, the role of a school Physics teacher is pivotal in generating interest amongst students for the subject as otherwise the long –term implications at higher levels of study turn out to be very severe.

Deep Fuzzy K-Means With Adaptive Loss and Entropy Regularization

Neural network based clustering methods usually have better performance compared to the conventional approaches due to more efficient feature extraction. Most of existing deep clustering techniques either exploit graph information as prior to extract pivotal deep structure from the raw data and simply utilizes stochastic gradient descent (SGD). However, they often suffer from separating the learning steps regarding dimensionality reduction and clustering. To address these issues, a novel deep model named as deep fuzzy k-means (DFKM) with adaptive loss function and entropy regularization is proposed. DFKM performs deep feature extraction and fuzzy clustering simultaneously to generate a more appropriate nonlinear feature map. Additionally, DFKM incorporates FKM so that fuzzy information is utilized to represent a clear structure of deep clusters. To further promote the robustness of the model, a robust loss function is applied to the objective with adaptive weights. Moreover, an entropy regularization is employed for affinity to provide confidence of each assignment and the corresponding membership and centroid matrices are updated by close-form solutions rather than SGD. Extensive experiments show that DFKM has better performance compared to the state-of-the-art fuzzy clustering techniques under three clustering metrics.

Regularized Deep Clustering Method for Fault Trend Analysis

Effective fault feature extraction is the key of fault diagnosis. In previous works, it is shown that some embedding methods and unsupervised deep learning methods have the ability to extract fault features from raw signals directly, such as PCA and deep autoencoder. Particularly, deep autoencoder has been shown in relevant research that it caneffectively extract the hidden ‘trend’ associated with machinery health states which can be useddirectly for online anomaly detection and prediction. However, in practical online fault diagnosis, the discrimination between successive signals is small due to the slow degradation progress and the external noise. Therefore,it is important to optimize the feature extraction process to achieve better online fault tracking. In this paper, a regularized deep clustering algorithm is proposed to guide the optimization process of feature extraction which combines embedding method and semi-guided learning. A regularization term for the cluster center points is proposed to make the feature optimization convergein a monotonic linear trend. In order to verify the effectiveness of the method, an accelerated gearbox run-to-failure experiment is carried out. The result shows that the feature optimization method can optimize the fault features on the basis of the deep autoencoder algorithm in two aspects: a better distinction of the fault features in short term and a more consistent trend of the gear wear in the long term.

Method for Clustering of Brain Activity Data Derived from EEG Signals

A method for assessing separability of EEG signals associated with three classes of brain activity is proposed. The EEG signals are acquired from 23 subjects, gathered from a headset consisting of 14 electrodes. Data are processed by applying Discrete Wavelet Transform (DWT) for the signal analysis and an autoencoder neural network for the brain activity separation. Processing involves 74 wavelets from 3 DWT families: Coiflets, Daubechies and Symlets. Euclidean distance between clusters normalized with respect to the standard deviation of the whole set of data are used to separate each task performed by participants. The results of this stage allow for an assessment of separability between subsets of data associated with each activity performed by experiment participants. The speed of convergence of the training process employing deep learning-based clustering is also measured.

Unsupervised deep learning and semi-automatic data labeling in weed discrimination

In recent years, supervised Deep Neural Networks have achieved the state-of-the-art in image recognition and this success has spread in many areas. In agricultural field, several researches have been conducted using architectures such as Convolutional Neural Networks. Despite this success, these works are still highly dependent on very time–costly manual data labeling. In contrast to this scenario, Unsupervised Deep Learning has no dependency on data labeling and is targeted as the future of the area, but after a promising start has been obfuscated by the success of supervised networks. Meanwhile, the low-cost of acquisition of field crop imagery using Unnamed Aerial Vehicles could be largely boosted in real-world applications if these images could be annotated without the need for a human specialist. In this work, we tested two recent unsupervised deep clustering algorithms, Joint Unsupervised Learning of Deep Representations and Image Clusters (JULE) and Deep Clustering for Unsupervised Learning of Visual Features (DeepCluster), using two public weed datasets. The first dataset was captured in a soybean plantation in Brazil and discriminates weeds between grass and broadleaf. The second dataset consists of 17,509 labeled images of eight nationally significant weed species native to Australia. We evaluated the purely unsupervised clustering performance using the NMI and Unsupervised Clustering Accuracy metrics and analysed the effects of techniques like data augmentation and transfer learning to improve clustering quality in a broad discussion that can be useful for unsupervised deep clustering in general. We also propose the usage of semi-automatic data labeling which greatly reduces the cost of manual data labeling and can be easily replicated to different datasets. This approach achieved 97% accuracy in discrimination of grass and broadleaf while reducing the number of manual annotations by 100 times, using a custom set of training images, without images labeled using inaccurate clusters.

Microseismic Focal Mechanisms and Implications for Changes in Stress during the 2014 Newberry EGS Stimulation

Abstract We adapt the relative polarity method from Shelly et al. (2016) to compute focal mechanisms for microearthquakes associated with the 2014 hydroshearing stimulation at the Newberry volcano geothermal site. We focus the analysis on events relocated by Aguiar and Myers (2018), who report that six event clusters predominantly comprise the 2014 sequence. Data quality allows focal mechanism analysis for four of the six event clusters. We use Hardebeck and Shearer (2002, 2003; hereafter HASH) to compute focal mechanisms based on first‐motion polarities and S/P amplitude ratios. We manually determine P‐ and S‐wave polarities for a well‐recorded reference event in each cluster, then use waveform cross correlation to determine whether recordings of other events in the cluster are the same or reversed polarity at each network station. Most waveform polarities are consistent with the affiliated reference event, indicating similar focal mechanisms within each cluster. The deeper clusters are east–west‐striking normal faults, whereas the shallower clusters, close to the top of the open‐hole section of the borehole, are strike slip with east–west motion. Regional studies and prestimulation borehole breakouts find the maximum stress direction is vertical and maximum horizontal stress is approximately north–south. Fault geometry and focal mechanisms of microseismicity during the stimulation suggest that increased pressure from fluid injection predominantly caused changes in horizontal stress, consistent with predictions from numerical studies of stress change caused by fluid injection. At shallow depths, where previous studies suggest the difference between vertical and horizontal stress is lowest, injection appears to have rotated the direction of maximum stress from vertical to horizontal, resulting in strike‐slip motion. At greater depth, vertical stress continued to be the dominant direction during the stimulation, but fault orientation indicates either reactivation of pre‐existing fractures or rotation of the direction of maximum horizontal stress from approximately north–south to east–west.

Integration of Neural Networks and Probabilistic Spatial Models for Acoustic Blind Source Separation

We formulate a generic framework for blind source separation (BSS), which allows integrating data-driven spectro-temporal methods, such as deep clustering and deep attractor networks, with physically motivated probabilistic spatial methods, such as complex angular central Gaussian mixture models. The integrated model exploits the complementary strengths of the two approaches to BSS: the strong modeling power of neural networks, which, however, is based on supervised learning, and the ease of unsupervised learning of the spatial mixture models whose few parameters can be estimated on as little as a single segment of a real mixture of speech. Experiments are carried out on both artificially mixed speech and true recordings of speech mixtures. The experiments verify that the integrated models consistently outperform the individual components. We further extend the models to cope with noisy, reverberant speech and introduce a cross-domain teacher–student training where the mixture model serves as the teacher to provide training targets for the student neural network.

A Deep Clustering Algorithm based on Gaussian Mixture Model

Clustering autonomously learns the implicit cluster structure in the original data without prior knowledge. The effect of ordinary clustering algorithms is not good to cluster high-dimensional data. In this paper, we propose a deep clustering algorithm based on Gaussian mixture model, which combines two models of stacked auto-encoder and Gaussian mixture model. This algorithm uses the expectation maximization algorithm of reducing dimension data feature to train Gaussian mixture and updates the data cluster so that the data is clustered in the feature space. The experimental results demonstrate that the proposed algorithm improves the clustering accuracy, and verifies the effectiveness of the algorithm.

Abstract 1635: A scalable, cloud-based, unsupervised deep learning system for identification, extraction, and summarization of potentially imperceptible patterns in whole-slide images of breast cancer tissue

Abstract Digital pathology images potentially contain novel patterns that may be perceived by modern deep learning models, but not humans. Prior unsupervised pattern recognition approaches have been used to reveal prognostically-relevant subtypes of glioblastoma (PMID: 28984190) and breast density segmentation (PMID: 26915120), and may complement supervised machine learning models trained using labeled data. In the Cancer Prevention Study II (CPS-II) cohort (PMID: 12015775), high-resolution, digitized hemotoxylin and eosin diagnostic slides are available for approximately 1,700 breast cancer cases providing an opportunity to perform unsupervised pattern recognition image analysis for epidemiologic breast cancer studies. Given the size of the dataset and complexity of the models, we constructed an end-to-end analytical pipeline, including preprocessing, feature engineering, and clustering, using cloud-based technologies that enable analysis at scale. Prior to training the unsupervised models, we faced issues converting raw images with open-source software. Specifically, OpenSlides could not open the Leica Versa SCN files due to their proprietary format while BioFormats inverted colors. To fix these issues, we altered the BioFormats library to successfully convert the files into a TIFF format. Since this issue likely affects other researchers, we are in discussions to provide the fix under a public license. TIFF formatted images were then denoised through color normalization to reduce hue variance and artifact detection to remove unwanted features such as pathologist annotations. Due to the computational complexity of analyzing the full image, images were padded with white space to ensure divisibility and broken into nine tiles of a predefined size. To further reduce computation time, uninformative tiles were filtered based on a predetermined threshold of artifact and white space composition. The remaining tiles were input to the unsupervised models. We used convolutional autoencoders, specifically a modified VGG-16 model without pretrained weights and a deep embedded clustering algorithm. These models learn representations of the images called ‘feature vectors’ and encode the images’ salient patterns. The final model was chosen based on iterative testing on a subsample of 100 images (N=21,472 tiles) and performance comparison of various VGG-inspired autoencoders. The feature vectors were clustered by K-means to summarize the information in a format suitable for statistical analyses. Our initial results show that the system captures macro-scale tissue patterns at lower magnifications (1x and 5x) and produces clusters that can be integrated into epidemiological studies of breast cancer etiology and prognosis. Citation Format: Jacob L. Evans, William Seo, Mary Macheski-Preston, Michelle Fritz, Samantha Puvanesarajah, James Hodge, Ted Gansler, Susan Gapstur, Mia M. Gaudet, Michelle Yi. A scalable, cloud-based, unsupervised deep learning system for identification, extraction, and summarization of potentially imperceptible patterns in whole-slide images of breast cancer tissue [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2019; 2019 Mar 29-Apr 3; Atlanta, GA. Philadelphia (PA): AACR; Cancer Res 2019;79(13 Suppl):Abstract nr 1635.

Pathway-based deep clustering for molecular subtyping of cancer.

Cancer is a genetic disease comprising multiple subtypes that have distinct molecular characteristics and clinical features. Cancer subtyping helps in improving personalized treatment and making decision, as different cancer subtypes respond differently to the treatment. The increasing availability of cancer related genomic data provides the opportunity to identify molecular subtypes. Several unsupervised machine learning techniques have been applied on molecular data of the tumor samples to identify cancer subtypes that are genetically and clinically distinct. However, most clustering methods often fail to efficiently cluster patients due to the challenges imposed by high-throughput genomic data and its non-linearity. In this paper, we propose a pathway-based deep clustering method (PACL) for molecular subtyping of cancer, which incorporates gene expression and biological pathway database to group patients into cancer subtypes. The main contribution of our model is to discover high-level representations of biological data by learning complex hierarchical and nonlinear effects of pathways. We compared the performance of our model with a number of benchmark clustering methods that recently have been proposed in cancer subtypes. We assessed the hypothesis that clusters (subtypes) may be associated to different survivals by logrank tests. PACL showed the lowest p-value of the logrank test against the benchmark methods. It demonstrates the patient groups clustered by PACL may correspond to subtypes which are significantly associated with distinct survival distributions. Moreover, PACL provides a solution to comprehensively identify subtypes and interpret the model in the biological pathway level. The open-source software of PACL in PyTorch is publicly available at https://github.com/tmallava/PACL.

Unsupervised Attention Embedding for Document Clustering

Deep clustering algorithms perform learning feature representations and clustering tasks jointly by using neural networks with significantly improved performance over the traditional k-means or spectral clustering. Some groundbreaking proposals extract data spaces directly in “bags of words” approach without considering the semantic information of each document as inputs to deep auto-encoder networks. But these algorithms suffer from inaccurate feature space from the encoder output when dealing with incomprehensible and high-dimensional data. For solving this problem in this paper, an Attention-based Deep Embedded Clustering (ADEC) algorithm is proposed to improve representation of data space. ADEC extracts high quality embedded features and performs clustering jointly with learning embedded features which are suitable for document clustering. The experimental result shows that the performance and accuracy of document clustering is improved significantly using the ADEC clustering framework on two datasets REUTERS-10K and REUTERS.

DeepBouton: Automated Identification of Single-Neuron Axonal Boutons at the Brain-Wide Scale.

Fine morphological reconstruction of individual neurons across the entire brain is essential for mapping brain circuits. Inference of presynaptic axonal boutons, as a key part of single-neuron fine reconstruction, is critical for interpreting the patterns of neural circuit wiring schemes. However, automated bouton identification remains challenging for current neuron reconstruction tools, as they focus mainly on neurite skeleton drawing and have difficulties accurately quantifying bouton morphology. Here, we developed an automated method for recognizing single-neuron axonal boutons in whole-brain fluorescence microscopy datasets. The method is based on deep convolutional neural networks and density-peak clustering. High-dimensional feature representations of bouton morphology can be learned adaptively through convolutional networks and used for bouton recognition and subtype classification. We demonstrate that the approach is effective for detecting single-neuron boutons at the brain-wide scale for both long-range pyramidal projection neurons and local interneurons.

Observations of guided waves from the Pamir seismic zone provide additional evidence for the existence of subducted continental lower crust

As part of the TIPAGE (TIen shan – PAmir GEodynamic program) project, passive seismological observations were made along an approximately N-S profile crossing the Pamir seismic zone for about one year. From these observations guided waves were recognized. These guided waves occur as a single, continuous, secondary, compressional (P) wave phase behind the first P-wave arrivals. An equivalent phase in the shear (S) wavefield is hardly recognizable. Modelling of the phase shows that an approximately 10 km thick low velocity zone (LVZ) between the Moho and about 160 km depth reproduces the guided waves as a single, continuous phase much better than a 15–20 km thick LVZ. Modelling of the arrival times of the guided waves reveals that a model with a P-wave velocity of 6.3 km/s above about 100 km depth, and a velocity of 7.6 km/s between this depth and the deep cluster of earthquakes at about 150 km depth provides the best fit to the observed travel-time data. One plausible way to explain the low velocity of 6.3 km/s is to invoke the presence of melts in the LVZ. Then, taking a velocity of 6.9 km/s for the lower crust being subducted, about 10–13% melt is required to obtain a velocity of about 6.3 km/s in the LVZ between the Moho and about 100 km depth. This would be in keeping with the estimated burial depths from xenoliths of Gondwana terrane affinity brought to the surface in the southeastern Pamir around 11 million yr. ago. The present-day LVZ is interpreted to comprise continental lower crust. Although guided waves are known to exist associated with subducted oceanic crust or fault zones, this is the first time to the knowledge of the authors that guided waves have been observed resulting from a LVZ associated with subducted continental lower crust.

WITHDRAWN: Adaptive method of micro learning cell based on AP deep clustering

This article has been withdrawn at the request of the authors(s) and/or editor. The full article has been removed because it was published without the consent of all authors listed. The Publisher apologizes for any inconvenience this may cause. The full Elsevier Policy on Article Withdrawal can be found at https://www.elsevier.com/about/our-business/policies/article-withdrawal

Deep divergence-based approach to clustering

A promising direction in deep learning research consists in learning representations and simultaneously discovering cluster structure in unlabeled data by optimizing a discriminative loss function. As opposed to supervised deep learning, this line of research is in its infancy, and how to design and optimize suitable loss functions to train deep neural networks for clustering is still an open question. Our contribution to this emerging field is a new deep clustering network that leverages the discriminative power of information-theoretic divergence measures, which have been shown to be effective in traditional clustering. We propose a novel loss function that incorporates geometric regularization constraints, thus avoiding degenerate structures of the resulting clustering partition. Experiments on synthetic benchmarks and real datasets show that the proposed network achieves competitive performance with respect to other state-of-the-art methods, scales well to large datasets, and does not require pre-training steps.

Combining Spectral and Spatial Features for Deep Learning Based Blind Speaker Separation

This study tightly integrates complementary spectral and spatial features for deep learning based multi-channel speaker separation in reverberant environments. The key idea is to localize individual speakers so that an enhancement network can be trained on spatial as well as spectral features to extract the speaker from an estimated direction and with specific spectral structures. The spatial and spectral features are designed in a way such that the trained models are blind to the number of microphones and microphone geometry. To determine the direction of the speaker of interest, we identify time-frequency T-F units dominated by that speaker and only use them for direction estimation. The T-F unit level speaker dominance is determined by a two-channel chimera++ network, which combines deep clustering and permutation invariant training at the objective function level, and integrates spectral and interchannel phase patterns at the input feature level. In addition, T-F masking based beamforming is tightly integrated in the system by leveraging the magnitudes and phases produced by beamforming. Strong separation performance has been observed on reverberant talker-independent speaker separation, which separates reverberant speaker mixtures based on a random number of microphones arranged in arbitrary linear-array geometry.

Archaeal ammonia oxidation plays a part in late Quaternary nitrogen cycling in the South China Sea

Thaumarchaeota, as an ammonia oxidizing archaea (AOA), is crucial for modern marine nitrogen cycling; however, little is known about its history during the Quaternary climate change. Here, isoprenoidal glycerol dialkyl glycerol tetraethers (GDGTs), biomarkers of Thaumarchaeota, were used to trace the role of AOA in the South China Sea (SCS) for the past 160 kyr. The GDGT-[2]/[3] ratio was firstly argued as an indicator of contribution of shallow Thaumarchaeota cluster that is more active in ammonia oxidization (AO) than the deep cluster, and then, was used to reconstruct AO in the past. The inferred AO exhibited intensification in the interglacials, and moreover, showed strong precessional cycles with enhancements at the precessional maxima when boreal winter insolation was the highest. The AOA record varied in line with isotope record of organic nitrogen (δ15Norg) that is modulated by the strength of diazotroph N2 fixation (NF), suggesting a close coupling of increased AO with enhanced NF during periods of weak east Asian winter monsoon (EAWM) and hence increase of upper water stratification. AO intensification, a step of a series of dissolved oxygen consuming processes, is hereby hypothesized to encourage NF when the EAWM weakens. This result might be a reference for the future NF trend in the current situation of enhanced ocean deoxygenation due to global warming.

Adaptive Self-paced Deep Clustering with Data Augmentation

Deep clustering gains superior performance than conventional clustering by jointly performing feature learning and cluster assignment. Although numerous deep clustering algorithms have emerged in various applications, most of them fail to learn robust cluster-oriented features which in turn hurts the final clustering performance. To solve this problem, we propose a two-stage deep clustering algorithm by incorporating data augmentation and self-paced learning. Specifically, in the first stage, we learn robust features by training an autoencoder with examples that are augmented by random shifting and rotating the given clean examples. Then, in the second stage, we encourage the learned features to be cluster-oriented by alternatively finetuning the encoder with the augmented examples and updating the cluster assignments of the clean examples. During finetuning the encoder, the target of each augmented example in the loss function is the center of the cluster to which the clean example is assigned. The targets may be computed incorrectly, and the examples with incorrect targets could mislead the encoder network. To stabilize the network training, we select most confident examples in each iteration by utilizing the adaptive self-paced learning. Extensive experiments validate that our algorithm outperforms the state of the arts on four image datasets.

Deep Clustering-Based Codebook Design for Massive MIMO Systems

Deep learning (DL) can effectively handle explosive data and solve the complicated non-convex problems. Recently, it is used to reduce the overhead of channel state information (CSI) feedback and improve the performance of the limited feedback massive MIMO systems. However, the complex wireless environment and the high-dimensional data of massive MIMO channel degrades the performance of the DL based codebook design. To address this issue, we propose a deep clustering (DC) based codebook design for massive MIMO systems. At first, deep neural network (DNN) learns the key propagation characteristics of the wireless channel according to the training model. And then, the clustering algorithm outputs the centroids of each key propagation characteristic, which can be used to reconstruct the valid space of massive MIMO channel. Finally, massive MIMO codebook is constructed by the valid space. Since the amount of the key propagation characteristics are far less than the dimension of massive MIMO channel, the proposed design can significantly simplify the parameters and the structures of DNN. Moreover, the clustering algorithm output the information with minimum sum-distance with the real channel sequences which can acquire the maximum sum-rate of massive MIMO codebook design. The robustness of the proposed codebook design is validated by carrying out extensive simulation in various scenarios and antenna array structures. Simulation results consist with theoretical analysis in terms of the achievable rate, and demonstrate that the proposed codebook design outperforms the traditional schemes.