Clusters In Feature Space Research Articles

Riemannian Manifold-Based Feature Space and Corresponding Image Clustering Algorithms.

Image feature representation is a key factor influencing the accuracy of clustering. Traditional point-based feature spaces represent spectral features of an image independently and introduce spatial relationships of pixels in the image domain to enhance the contextual information expression ability. Mapping-based feature spaces aim to preserve the structure information, but the complex computation and the unexplainability of image features have a great impact on their applications. To this end, we propose an explicit feature space called Riemannian manifold feature space (RMFS) to present the contextual information in a unified way. First, the Gaussian probability distribution function (pdf) is introduced to characterize the features of a pixel in its neighborhood system in the image domain. Then, the feature-related pdfs are mapped to a Riemannian manifold, which constructs the proposed RMFS. In RMFS, a point can express the complex contextual information of corresponding pixel in the image domain, and pixels representing the same object are linearly distributed. This gives us a chance to convert nonlinear image segmentation problems to linear computation. To verify the superiority of the expression ability of the proposed RMFS, a linear clustering algorithm and a fuzzy linear clustering algorithm are proposed. Experimental results show that the proposed RMFS-based algorithms outperform their counterparts in the spectral feature space and the RMFS-based ones without the linear distribution characteristics. This indicates that the RMFS can better express features of an image than spectral feature space, and the expressed features can be easily used to construct linear segmentation models.

Point Cloud Classification Using Content-Based Transformer via Clustering in Feature Space

Recently, there have been some attempts of Transformer in 3D point cloud classification. In order to reduce computations, most existing methods focus on local spatial attention, but ignore their content and fail to establish relationships between distant but relevant points. To overcome the limitation of local spatial attention, we propose a point content-based Transformer architecture, called PointConT for short. It exploits the locality of points in the feature space (content-based), which clusters the sampled points with similar features into the same class and computes the self-attention within each class, thus enabling an effective trade-off between capturing long-range dependencies and computational complexity. We further introduce an Inception feature aggregator for point cloud classification, which uses parallel structures to aggregate high-frequency and low-frequency information in each branch separately. Extensive experiments show that our PointConT model achieves a remarkable performance on point cloud shape classification. Especially, our method exhibits 90.3% Top-1 accuracy on the hardest setting of ScanObjectNN. Source code of this paper is available at https://github.com/yahuiliu99/PointConT.

Cross-Validation Strategy Impacts the Performance and Interpretation of Machine Learning Models

Abstract Machine learning algorithms are able to capture complex, nonlinear, interacting relationships and are increasingly used to predict agricultural yield variability at regional and national scales. Using explainable artificial intelligence (XAI) methods applied to such algorithms may enable better scientific understanding of drivers of yield variability. However, XAI methods may provide misleading results when applied to spatiotemporal correlated datasets. In this study, machine learning models are trained to predict simulated crop yield from climate indices, and the impact of cross-validation strategy on the interpretation and performance of the resulting models is assessed. Using data from a process-based crop model allows us to then comment on the plausibility of the “explanations” provided by XAI methods. Our results show that the choice of evaluation strategy has an impact on (i) interpretations of the model and (ii) model skill on held-out years and regions, after the evaluation strategy is used for hyperparameter tuning and feature selection. We find that use of a cross-validation strategy based on clustering in feature space achieves the most plausible interpretations as well as the best model performance on held-out years and regions. Our results provide the first steps toward identifying domain-specific “best practices” for the use of XAI tools on spatiotemporal agricultural or climatic data. Significance Statement “Explainable” or “interpretable” machine learning (XAI) methods have been increasingly used in scientific research to study complex relationships between climatic and biogeoscientific variables (such as crop yield). However, these methods can return contradictory, implausible, or ambiguous results. In this study, we train machine learning models to predict maize yield anomalies and vary the model evaluation method used. We find that the evaluation (cross validation) method used has an effect on model interpretation results and on the skill of resulting models in held-out years and regions. These results have implications for the methodological design of studies that aim to use XAI tools to identify drivers of, for example, crop yield variability.

Context Label Learning: Improving Background Class Representations in Semantic Segmentation.

Background samples provide key contextual information for segmenting regions of interest (ROIs). However, they always cover a diverse set of structures, causing difficulties for the segmentation model to learn good decision boundaries with high sensitivity and precision. The issue concerns the highly heterogeneous nature of the background class, resulting in multi-modal distributions. Empirically, we find that neural networks trained with heterogeneous background struggle to map the corresponding contextual samples to compact clusters in feature space. As a result, the distribution over background logit activations may shift across the decision boundary, leading to systematic over-segmentation across different datasets and tasks. In this study, we propose context label learning (CoLab) to improve the context representations by decomposing the background class into several subclasses. Specifically, we train an auxiliary network as a task generator, along with the primary segmentation model, to automatically generate context labels that positively affect the ROI segmentation accuracy. Extensive experiments are conducted on several challenging segmentation tasks and datasets. The results demonstrate that CoLab can guide the segmentation model to map the logits of background samples away from the decision boundary, resulting in significantly improved segmentation accuracy. Code is available.

Integrated Test Pattern Extraction and Generation for Accurate Lithography Modeling

A set of good test patterns is a key for accurate lithography modeling. We address three sub-problems in test pattern extraction: (1) Each pattern is associated with one or more gauges, where CD is measured and modeling is performed. We perform gauge clustering in feature space, while the goal is to minimize the number of chosen patterns. (2) Sample patterns do not usually guarantee enough coverage, so once coverage of chosen patterns is identified, some synthetic patterns are systematically generated to fill the empty region in feature space. (3) Pattern representation defines the feature space. We propose to use some Gaussian convolutions (or a few of their principal components) together with image parameter set (IPS) to reflect exposure for optical model and both PEB and development for resist model. Experiments with 1xnm DRAM design demonstrate that lithography modeling through our test pattern extraction and generation is performed with 35% smaller number of test patterns while CD error is reduced by 55%, with pattern extraction in feature space of IPS parameters as a reference.

Research on Big Data-Driven Rural Revitalization Sharing Cogovernance Mechanism Based on Cloud Computing Technology

The political arrangement and strategic action to realize socialist sharing in the new era is the rural revitalization strategy, which is a development strategy to complete the historical task of farmers living a better life and solve the problem of insufficient rural development. Data sharing and driving, on the other hand, remain the weak links in China’s digital village construction. This paper proposes to develop a BD (big data)-driven rural revitalization governance, sharing, and monitoring platform based on CC (cloud computing) technology to promote the reshaping of rural governance patterns and to provide systematic support for the implementation of rural revitalization strategies. Knowledge governance rules and processes for rural community public affairs are created in a BD environment. Clustering in feature space is considered to address the low efficiency caused by clustering algorithms in high-dimensional data. The method proposed in this paper is perfectly matched in both the preprocessing and clustering steps, which not only ensures the algorithm’s accuracy but also significantly improves its efficiency.

Approach of variable clustering and compression for learning large Bayesian networks

This paper describes a new approach for learning structures of large Bayesian networks based on blocks resulting from feature space clustering. This clustering is obtained using normalized mutual information. And the subsequent aggregation of blocks is done using classical learning methods except that they are input with compressed information about combinations of feature values for each block. Validation of this approach is done for Hill-Climbing as a graph enumeration algorithm for two score functions: BIC and MI. In this way, potentially parallelizable block learning can be implemented even for those score functions that are considered unsuitable for parallelizable learning. The advantage of the approach is evaluated in terms of speed of work as well as the accuracy of the found structures.

Feature Selection Based on Multi Resolution Analysis of Text Documents for Effective Clustering

Users of digital era are overwhelmed with large volumes of text collections. Most of the text collections are without class labels. Clustering is the only feasible solution to extract valuable insights from the data. Clustering of text collections in high dimensional space is inefficient. Several unsupervised dimensionality reduction methods have been proposed in the literature. Feature selection methods are easy to interpret. Filter feature selection methods have been proved to be scalable and efficient for high dimensional datasets. The aim of this work is to propose an unsupervised univariate filter feature selection method for efficient clustering of very high dimensional text datasets in low dimensional feature space. Wavelets are mathematical functions that can transform a signal into space or time, frequency domain to analyze the signal at different resolutions of transformed domain. Wavelet transforms are efficient in identifying transients of a signal. Stationary wavelet transform using Symlet of order 2 is used to identify the most discriminant features of text documents for efficient clustering in low dimensional feature space. The proposed feature selection method is compared with nine other relevant methods by their quality of clustering solution on seven real text document collections. The proposed method has been able to identify the most discriminative features that have resulted in the best peak clustering performance better than clustering performance with all the features, on four out of seven datasets, with at most 1.5% of top-rated features.

Machine learning algorithm for feature space clustering of mixed data with missing information based on molecule similarity

Clustering Algorithms have just fascinated significant devotion in machine learning applications owing to their great competence. Nevertheless, the existing algorithms quite have approximately disputes that need to be further deciphered. For example, most existing algorithms transform one type of feature into another type, which disregards the explicit possessions of information. In addition, most of them deliberate whole features, which may lead to difficulty in calculation and effect in sub-optimal presentation. To address the above difficulties, this paper proposes a novel technique for clustering categorical and numerical features based on feature space clustering of mixed data with missing information (FSCMMI). The procedure involves three stages. Initially, FSCMMI divides the given dataset depending on missing information in instances and features types. The second stage uses the decision-tree procedure to identify the association between instances. Finally, the third stage is used for computing the closeness measure for numerical features and categorical features. Meanwhile, we propose a new training algorithm to cluster mixed datasets. Extensive experimental results on benchmark datasets show that the proposed FSCMMI outperforms several state-of-art clustering methods in terms of accuracy and efficiency.

A new neurofeedback training method based on feature space clustering to control EEG features within target clusters

BackgroundWithin the most commonly used neurofeedback training methods, a threshold has been defined for each EEG feature wherein subjects’ status during training can be assessed according to the given value. In the present study, a neurofeedback training method based on feature-space clustering was proposed in order to assess subjects’ status more accurately. New methodNeural gas algorithm was employed for feature space clustering. Then, the clusters were labeled as initial clusters (where the EEG features were placed prior to training) and target (where the EEG features should be shifted towards during training) ones. A scoring index was defined whose value was determined according to subjects’ brain activity. This method was simulated in two versions: soft-boundary and hard-boundary based methods. ResultsThe results of the present simulation showed that the proposed hard-boundary based version could guide the subjects towards the boundaries of the target clusters and even their status would be stabilized in case of too many changes in subjects’ EEG features. In the proposed soft-boundary based version, in case of too many changes in training features, the subjects would not be encouraged and they could be guided towards the target boundaries. ConclusionThe proposed hard-boundary based version could be effective in guiding a subject towards being placed within the boundaries of target clusters and even beyond them if no specific limits exited for EEG features. As well, the soft-boundary based version could be useful when controlling EEG features within a limit.

Early Detection of Health Changes in the Elderly Using In-Home Multi-Sensor Data Streams

The rapid aging of the population worldwide requires increased attention from healthcare providers and the entire society. For the elderly to live independently, many health issues related to old age, such as frailty and risk of falling, need increased attention and monitoring. When monitoring daily routines for older adults, it is desirable to detect the early signs of health changes before serious health events, such as hospitalizations, happen so that timely and adequate preventive care may be provided. By deploying multi-sensor systems in homes of the elderly, we can track trajectories of daily behaviors in a feature space defined using the sensor data. In this article, we investigate a methodology for tracking the evolution of the behavior trajectories over long periods (years) using high-dimensional streaming clustering and provide very early indicators of changes in health. If we assume that habitual behaviors correspond to clusters in feature space and diseases produce a change in behavior, albeit not highly specific, tracking trajectory deviations can provide hints of early illness. Retrospectively, we visualize the streaming clustering results and track how the behavior clusters evolve in feature space with the help of two dimension-reduction algorithms: Principal Component Analysis and t-distributed Stochastic Neighbor Embedding. Moreover, our tracking algorithm in the original high-dimensional feature space generates early health warning alerts if a negative trend is detected in the behavior trajectory. We validated our algorithm on synthetic data and tested it on a pilot dataset of four TigerPlace residents monitored with a collection of motion, bed, and depth sensors over 10 years. We used the TigerPlace electronic health records to understand the residents’ behavior patterns and to evaluate the health warnings generated by our algorithm. The results obtained on the TigerPlace dataset show that most of the warnings produced by our algorithm can be linked to health events documented in the electronic health records, providing strong support for a prospective deployment of the approach.

Geometrical inspired pre-weighting enhances Markov clustering community detection in complex networks

Markov clustering is an effective unsupervised pattern recognition algorithm for data clustering in high-dimensional feature space. However, its community detection performance in complex networks has been demonstrating results far from the state of the art methods such as Infomap and Louvain. The crucial issue is to convert the unweighted network topology in a ‘smart-enough’ pre-weighted connectivity that adequately steers the stochastic flow procedure behind Markov clustering. Here we introduce a conceptual innovation and we discuss how to leverage network latent geometry notions in order to design similarity measures for pre-weighting the adjacency matrix used in Markov clustering community detection. Our results demonstrate that the proposed strategy improves Markov clustering significantly, to the extent that it is often close to the performance of current state of the art methods for community detection. These findings emerge considering both synthetic ‘realistic’ networks (with known ground-truth communities) and real networks (with community metadata), and even when the real network connectivity is corrupted by noise artificially induced by missing or spurious links. Our study enhances the generalized understanding of how network geometry plays a fundamental role in the design of algorithms based on network navigability.

Benchmark and application of unsupervised classification approaches for univariate data

Unsupervised machine learning, and in particular data clustering, is a powerful approach for the analysis of datasets and identification of characteristic features occurring throughout a dataset. It is gaining popularity across scientific disciplines and is particularly useful for applications without a priori knowledge of the data structure. Here, we introduce an approach for unsupervised data classification of any dataset consisting of a series of univariate measurements. It is therefore ideally suited for a wide range of measurement types. We apply it to the field of nanoelectronics and spectroscopy to identify meaningful structures in data sets. We also provide guidelines for the estimation of the optimum number of clusters. In addition, we have performed an extensive benchmark of novel and existing machine learning approaches and observe significant performance differences. Careful selection of the feature space construction method and clustering algorithms for a specific measurement type can therefore greatly improve classification accuracies.

Prototypical Network for Radar Image Recognition with Few Samples

At present, deep learning has surpassed humans in recognition tasks, but its performance strongly depends on sufficient training samples. When the training samples are extremely insufficient, overfitting also will damage the performance of network, despite the use of fine-tuning methods. One-shot learning (OSL) is proposed to overcome the difficulties of training neural networks while having almost no samples, which is a common situation in radar image recognition. This paper introduced OSL into recognition of some radar targets whose labelled images are seriously limited, and proposed an OSL method with prototypical networks based on the assumption that functions of the network trained with auxiliary samples also affects samples to be identified. The method of this paper involves image preprocessing, building of linearly separable feature space and feature vector clustering. Experiments on MSTAR dataset show that the performance of our method is better than deep transfer learning when the training samples in the target domain are seriously deficient.

FS-MLC: Feature selection for multi-label classification using clustering in feature space

Multi-label classification (MLC) has attracted many researchers in the field of machine learning as it has a straightforward problem statement with varied solution approaches. Multi-label classifiers predict multiple labels for a single instance. The problem becomes challenging with the increasing number of features, especially when there are many features and labels which depend on each other. It requires dimensionality reduction before applying any multi-label learning method. This paper introduces a method named FS-MLC (Feature Selection forMulti-Label classification using Clustering in feature-space). It is a wrapper feature selection method that uses clustering to find the similarity among features and example-based precision and recall as the metrics for feature rankings to improve the performance of the associated classifier in terms of sample-based measures. First, clusters are created for features considering them as instances then features from different clusters are selected as the representative of all the features for that cluster. It reduces the number of features as a single feature represents multiple features within a cluster. It neither requires any parameter tuning nor the user threshold for the number of features selected. Extensive experimentation is performed to evaluate the efficacy of these reduced features using nine benchmark MLC datasets on twelve performance measures. The results show an impressive improvement in terms of sample-based precision, recall, and f1-score with up to 23%-93% discarded features.

X-ray study of spatial structures in Tycho’s supernova remnant using unsupervised deep learning

ABSTRACT Recent rapid development of deep learning algorithms, which can implicitly capture structures in high-dimensional data, opens a new chapter in astronomical data analysis. We report here a new implementation of deep learning techniques for X-ray analysis. We apply a variational autoencoder (VAE) using a deep neural network for spatio-spectral analysis of data obtained by Chandra X-ray Observatory from Tycho’s supernova remnant (SNR). We established an unsupervised learning method combining the VAE and a Gaussian mixture model (GMM), where the dimensions of the observed spectral data are reduced by the VAE, and clustering in feature space is performed by the GMM. We found that some characteristic spatial structures, such as the iron knot on the eastern rim, can be automatically recognized by this method, which uses only spectral properties. This result shows that unsupervised machine learning can be useful for extracting characteristic spatial structures from spectral information in observational data (without detailed spectral analysis), which would reduce human-intensive preprocessing costs for understanding fine structures in diffuse astronomical objects, e.g. SNRs or galaxy clusters. Such data-driven analysis can be used to select regions from which to extract spectra for detailed analysis and help us make the best use of the large amount of spectral data available currently and arriving in the coming decades.

Mining latent patterns in geoMobile data via EPIC

We coin the term geoMobile data to emphasize datasets that exhibit geo-spatial features reflective of human behaviors. We propose and develop an EPIC framework to mine latent patterns from geoMobile data and provide meaningful interpretations: we first ‘E’xtract latent features from high dimensional geoMobile datasets via Laplacian Eigenmaps and perform clustering in this latent feature space; we then use a state-of-the-art visualization technique to ‘P’roject these latent features into 2D space; and finally we obtain meaningful ‘I’nterpretations by ‘C’ulling cluster-specific significant feature-set. We illustrate that the local space contraction property of our approach is most superior than other major dimension reduction techniques. Using diverse real-world geoMobile datasets, we show the efficacy of our framework via three case studies.

Deep domain adaptation with manifold aligned label transfer

We propose a novel deep learning domain adaptation method that performs transductive learning from the source domain to the target domain based on cluster matching between the source and target features. The proposed method combines Adaptive Batch Normalization and Locality Preserving Projection-based subspace alignment on deep features to produce a common feature space for label transfer. Adaptive Batch Normalization automatically conditions the features from the source/target domain by normalizing the activations in each layer of our network. Following Manifold Subspace Alignment, we cluster the data in each domain using Gaussian Mixture Model clustering in feature space. The clusters are matched between domains to transfer labels from the closest source cluster to each target cluster. The transfer labels are compared to the network prediction, and the samples with consistent labels are used to adapt the network on the target domain. The proposed manifold-guided label transfer method produces state-of-the-art results for deep adaptation on digit recognition datasets. Furthermore, we perform domain adaptation on remote sensing datasets.

Robust Cell Detection and Segmentation for Image Cytometry Reveal Th17 Cell Heterogeneity.

Image cytometry enables quantitative cell characterization with preserved tissue architecture; thus, it has been highlighted in the advancement of multiplex immunohistochemistry (IHC) and digital image analysis in the context of immune-based biomarker monitoring associated with cancer immunotherapy. However, one of the challenges in the current image cytometry methodology is a technical limitation in the segmentation of nuclei and cellular components particularly in heterogeneously stained cancer tissue images. To improve the detection and specificity of single-cell segmentation in hematoxylin-stained images (which can be utilized for recently reported 12-biomarker chromogenic sequential multiplex IHC), we adapted a segmentation algorithm previously developed for hematoxlin and eosin-stained images, where morphological features are extracted based on Gabor-filtering, followed by stacking of image pixels into n-dimensional feature space and unsupervised clustering of individual pixels. Our proposed method showed improved sensitivity and specificity in comparison with standard segmentation methods. Replacing previously proposed methods with our method in multiplex IHC/image cytometry analysis, we observed higher detection of cell lineages including relatively rare TH 17 cells, further enabling sub-population analysis into TH 1-like and TH 2-like phenotypes based on T-bet and GATA3 expression. Interestingly, predominance of TH 2-like TH 17 cells was associated with human papilloma virus (HPV)-negative status of oropharyngeal squamous cell carcinoma of head and neck, known as a poor-prognostic subtype in comparison with HPV-positive status. Furthermore, TH 2-like TH 17 cells in HPV-negative head and neck cancer tissues were spatiotemporally correlated with CD66b+ granulocytes, presumably associated with an immunosuppressive microenvironment. Our cell segmentation method for multiplex IHC/image cytometry potentially contributes to in-depth immune profiling and spatial association, leading to further tissue-based biomarker exploration. © 2019 International Society for Advancement of Cytometry.

Self-adaptive separation of multiple partial discharge sources based on optimized feature extraction of cumulative energy function

Partial discharge (PD) measurement is an effective tool for insulation condition assessment of high-voltage power equipment. The occurrence of multiple PD sources causes great difficulty on pattern recognition and failure risk assessment. This paper presents a self-adaptive PD separation algorithm based on optimized feature extraction of cumulative energy (CE) function. The CE functions in time domain (TCE) and frequency domain (FCE) are calculated from PD waveforms and their FFT spectrums, respectively. By using an oblique line to cross the CE curves, width features are extracted from the intersection points between them. Through the mathematical morphology gradient (MMG) operation, sharpness features are extracted to characterize the rise steepness of CE. It is found that the separation capability of width and sharpness are dependent on the pre-selected oblique line and the structure element length (SEL) in MMG, respectively. In order to obtain satisfactory PD separation results for various experimental conditions, a density-function based parameter is proposed to evaluate the separation capability, and the oblique line and SEL are optimized with the goal of maximizing the evaluation parameter. A clustering algorithm is adopted to discover different clusters in feature space and separate PD signals. The separation algorithm is examined with mixed PD current pulses and ultra-high-frequency (UHF) signals acquired from experiments in laboratory and on-site equipment. The results indicate that the self-adaptive separation method is immune to the change of experimental conditions, and is effective for separating mixed PD signals.