Model-based Clustering Approach Research Articles

A game-inspired algorithm for marginal and global clustering

An often overlooked pitfall of model-based clustering is that it typically results in the same number of clusters per margin, an assumption that may not be natural in practice. We develop a clustering method that takes advantage of the sturdiness of model-based clustering, while attempting to mitigate this issue. The proposed approach allows each margin to have a varying number of clusters and employs a strategy game-inspired algorithm, named ‘Reign-and-Conquer’, to cluster the data. Since the proposed clustering approach only specifies a model for the margins, but leaves the joint unspecified, it has the advantage of being partially parallelizable; hence, the proposed approach is computationally appealing as well as more tractable for moderate to high dimensions than a ‘full’ (joint) model-based clustering approach. A battery of numerical experiments on simulated data indicates an overall good performance of the proposed methods in a variety of scenarios, and real datasets are used to showcase their usefulness in practice.

Sparse model-based clustering of three-way data via lasso-type penalties

Mixtures of matrix Gaussian distributions provide a probabilistic framework for clustering continuous matrix-variate data, which are increasingly common in various fields. Despite their widespread use and successful applications, these models suffer from over-parameterization, making them not suitable for even moderately sized matrix-variate data. To address this issue, we introduce a sparse model-based clustering approach for three-way data. Our approach assumes that the matrix mixture parameters are sparse and have different degrees of sparsity across clusters, enabling the induction of parsimony in a flexible manner. Estimation relies on the maximization of a penalized likelihood, with specifically tailored group and graphical lasso penalties. These penalties facilitate the selection of the most informative features for clustering three-way data where variables are recorded over multiple occasions, as well as allowing the identification of cluster-specific association structures. We conduct extensive testing of the proposed methodology on synthetic data and validate its effectiveness through an application to time-dependent crime patterns across multiple U.S. cities. Supplementary files for this article are available online.

An autoencoder-based confederated clustering leveraging a robust model fusion strategy for federated unsupervised learning

Concerns related to data privacy, security, and ethical considerations become more prominent as data volumes continue to grow. In contrast to centralized setups, where all data is accessible at a single location, model-based clustering approaches can be successfully employed in federated settings. However, this approach to clustering in federated settings is still relatively unexplored and requires further attention. As federated clustering deals with remote data and requires privacy and security to be maintained, it poses particular challenges as well as possibilities. While model-based clustering offers promise in federated environments, a robust model aggregation method is essential for clustering rather than the generic model aggregation method like Federated Averaging (FedAvg). In this research, we proposed an autoencoder-based clustering method by introducing a novel model aggregation method FednadamN, which is a fusion of Adam and Nadam optimization approaches in a federated learning setting. Therefore, the proposed FednadamN adopted the adaptive learning rates based on the first and second moments of gradients from Adam which offered fast convergence and robustness to noisy data. Furthermore, FednadamN also incorporated the Nesterov-accelerated gradients from Nadam to further enhance the convergence speed and stability. We have studied the performance of the proposed Autoencoder-based clustering methods on benchmark datasets and using the novel FednadamN model aggregation strategy. It shows remarkable performance gain in federated clustering in comparison to the state-of-the-art.

Dense SNP-based analyses complement forensic anthropology biogeographical ancestry assessments

Identification of unidentified human remains (UHRs) is crucial yet challenging, especially with traditional forensic techniques. Forensic anthropological examinations can yield ancestry estimations; however, the utility of these estimates is limited by the data points that can be collected from partial remains, complexities of admixture, and variation of phenotypic expression due to environmental effects. While it is generally known that anthropological estimates can be imprecise, the performance of these methods has not been studied at scale. Genome-wide SNP testing is an orthogonal approach for estimating ancestry and offers a unique opportunity to measure the magnitude of anthropological ancestry misattribution. Genomic ancestry inference leverages principal component analysis (PCA) and model-based clustering approaches. This study compares anthropologically determined ancestry with those estimated using genome-wide SNP markers. A dataset of 611 UHR samples with publicly available ancestry assessments from National Missing and Unidentified Persons System (NamUs) was analyzed. The genetic ancestry approach, validated against reference population samples, offers robust ancestry calculations for major population groups. Inconsistency between anthropological and genomic ancestry assignments were observed, particularly for admixed populations. Although forensic anthropological examinations remain valuable, their limitations emphasize the need for refinement and enhancement through the augmentation of SNP-based analyses. Further validation studies are crucial to define the uncertainty associated with both anthropological and genome-based ancestry estimates to resolve cases and aid law enforcement investigations. Additionally, current policy and practices for reporting ancestry for UHRs should be revisited to reduce potential misinformation.

Characteristics of limb kinematics in the gait disorders of post-stroke patients

Post-stroke gait disorders involve altered lower limb kinematics. Recently, the endpoint of the lower limb has been used as a control variable to understand gait kinematics better. In a cross-sectional study of sixty-seven post-stroke patients, the limb extension angle and effective limb length during gait were used as input variables with a mixed Gaussian model-based probabilistic clustering approach to identify five distinct clusters. Each cluster had unique characteristics related to motor paralysis, spasticity, balance ability, and gait strategy. Cluster 1 exhibited high limb extension angle and length values, indicating increased spasticity. Cluster 2 had moderate extension angles and high limb lengths, indicating increased spasticity and reduced balance ability. Cluster 3 had low limb extension angles and high limb length, indicating reduced balance ability, more severe motor paralysis, and increased spasticity. Cluster 4 demonstrated high extension angles and short limb lengths, with a gait strategy that prioritized stride length in the component of gait speed. Cluster 5 had moderate extension angles and short limb lengths, with a gait strategy that prioritized cadence in the component of gait speed. These findings provide valuable insights into post-stroke gait impairment and can guide the development of personalized and effective rehabilitation strategies.

Model-based clustering of multipath propagation in powerline communication channels

Powerline communication (PLC) channels are known to exhibit multipath propagation behaviour. The authors present a model-based framework to address the challenge of clustering multipath propagation components (MPCs) in PLC channels for indoor low-voltage (LV) environments. The framework employs a range of finite-mixture models (FMMs), including the gamma mixture model, the inverse gamma mixture model, the Gaussian mixture model, the inverse Gaussian mixture model, the Nakagami mixture model, the inverse Nakagami mixture model (INMM) and the Rayleigh mixture model, to identify clusters of MPCs. A measurement campaign of an unknown indoor LV PLC channel is conducted to obtain a channel response. From the channel response, the delay and magnitude parameters of the MPCs are extracted using the space-alternating generalised expectation maximisation algorithm adopted only for these parameters. A maximum likelihood approach and the expectation–maximisation algorithm are employed to fit the FMMs to the MPC delay-magnitude dataset to cluster MPCs in the delay domain. The results of the model-fitting process are then evaluated using the corrected Akaike information criterion (AICc), which enables a fair comparison of the candidate models over the feasible and finite range of clusters. A novel algorithm is introduced for estimating the feasible and finite range of clusters using the extracted delay and magnitude MPC parameters. The AICc’s ranking results show that the INMM model provides the best fit. Davies–Bouldin (DB) and Calinski–Harabasz (CH) indexes are used to compare the model-based clustering approach to the conventional distance-based clustering methods. Validation results show that CH and DB indexes closely agree in the optimal number of MPC clusters for model-based clustering, which corresponds to the most within-cluster compactness of MPCs and to the most between-cluster separation in the delay domain.

Remarkable variation in androecial morphology is closely associated with corolla traits in Western Hemisphere Justiciinae (Acanthaceae: Justicieae).

Few studies of angiosperms have focused on androecial evolution in conjunction with evolutionary shifts in corolla morphology and pollinator relationships. The Western Hemisphere clade of Justiciinae (Acanthaceae) presents the rare opportunity to examine remarkable diversity in staminal morphology. We took a phylogenetically informed approach to examine staminal diversity in this hypervariable group and asked whether differences in anther thecae separation is associated with phylogenetically informed patterns of variation in corolla morphology. We further discuss evidence for associations between anther diversity and pollinators in this lineage. For the Dianthera/Sarotheca/Plagiacanthus (DSP) clade of Western Hemisphere Justiciinae, we characterized floral diversity based on a series of corolla measurements and using a model-based clustering approach. We then tested for correlations between anther thecae separation and corolla traits, and for shifts in trait evolution, including evidence for convergence. There is evolutionary vagility in corolla and anther traits across the DSP clade with little signal of phylogenetic constraint. Floral morphology clusters into four distinct groups that are, in turn, strongly associated with anther thecae separation, a novel result in Acanthaceae and, to our knowledge, across flowering plants. These cluster groups are marked by floral traits that strongly point to associations with pollinating animals. Specifically, species that are known or likely to be hummingbird pollinated have stamens with parallel thecae, whereas those that are likely bee or fly pollinated have stamens with offset, divergent thecae. Our results suggest that anther thecae separation is likely under selection in concert with other corolla characters. Significant morphological shifts detected by our analyses corresponded to putative shifts from insect to hummingbird pollination. Results from this study support the hypothesis that floral structures function in an integrated manner and are likely subject to selection as a suite. Further, these changes can be hypothesized to represent adaptive evolution.

Genetic diversity and structure of natural populations of Arbutus unedo across altitude, bioclimate and geographical origin as revealed by ISSR markers

In order to assess the genetic diversity and structure of Arbutus unedo species, 66 individuals’ trees from 11 natural populations representing its main geographical area in Morocco and belonging to different bioclimates and altitude were analyzed using 14 ISSR primers. A total of 142 bands were obtained, of which 134 were polymorphic with a polymorphism percentage of 94.06% and a polymorphism information content (PIC) of =0.47. The use of 134 markers revealed a high level of genetic variation within and between populations. The multi-locus values of Hs and Ht were 0.24 and 0.34, respectively. The overall AMOVA analysis showed that 79.30% of the total genetic variability was within populations and only 20.70% was between populations. A large genetic differentiation between populations was detected (FST=0.206) which could be attributed to restricted gene flow (Nm=0.97). The results of the hierarchical AMOVA, revealed very little genetic differentiation between altitudinal, bioclimatic and geographic groups (FCT=3.20%, FCT= 2.30% and FCT=2.10% respectively). Furthermore, geographic distances were revealed not correlated with genetic distances between populations (r=0.05, Mantel t-test=0.38, P=0.64) suggesting that isolation by distance has not played an important role in shaping the genetic structure of this species. The NJ dendrogram and Bayesian model-based clustering approach identified four groups of A. unedo independently of their bioclimate, geographic origin and altitude. The data obtained in this study could play a crucial role in establish efficient strategies for genetic resources conservation and to work out the scheme of breeding programs of A. unedo

Sparse tree-based clustering of microbiome data to characterize microbiome heterogeneity in pancreatic cancer.

There is a keen interest in characterizing variation in the microbiome across cancer patients, given increasing evidence of its important role in determining treatment outcomes. Here our goal is to discover subgroups of patients with similar microbiome profiles. We propose a novel unsupervised clustering approach in the Bayesian framework that innovates over existing model-based clustering approaches, such as the Dirichlet multinomial mixture model, in three key respects: we incorporate feature selection, learn the appropriate number of clusters from the data, and integrate information on the tree structure relating the observed features. We compare the performance of our proposed method to existing methods on simulated data designed to mimic real microbiome data. We then illustrate results obtained for our motivating data set, a clinical study aimed at characterizing the tumor microbiome of pancreatic cancer patients.

Aggregated functional data model applied on clustering and disaggregation of UK electrical load profiles

AbstractUnderstanding electrical energy demand at the consumer level plays an important role in planning the distribution of electrical networks and offering of off-peak tariffs, but observing individual consumption patterns is still expensive. On the other hand, aggregated load curves are normally available at the substation level. The proposed methodology separates substation aggregated loads into estimated mean consumption curves, called typical curves, including information given by explanatory variables. In addition, a model-based clustering approach for substations is proposed based on the similarity of their consumers’ typical curves and covariance structures. The methodology is applied to a real substation load monitoring dataset from the UK and tested in eight simulated scenarios.

Performance evaluation of automotive dealerships using grouped mixture of regressions

Finite Mixture of Regressions (FMR) are among the most widely used models for dealing with heterogeneity in regression problems. FMR is a model-based clustering approach that models the data by assigning individual observations to one of the K latent regression clusters. In some applications, it is desired to cluster groups of observations together rather than individually. We present an extension to the regular FMR, that we call the Grouped Mixture of Regressions (GMR), which allows for a known group structure among observations, in addition to their possibly unknown heterogeneity. The research is motivated by a large financial longitudinal dataset from a brand of automotive dealerships across the United States. The task was to cluster the dealers based on their financial performance and also improve performance prediction for individual dealers. We derive a fast and salable algorithm for estimating the model parameters using Expectation–Maximization (EM). We also show how the group structure can improve prediction by sharing information among members of each group, as reflected in the posterior predictive density under GMR. The performance of the approach is assessed using both synthetic data as well as the dealership data. Among our findings are the superior predictive performance of GMR and the ability to correctly detect the number of clusters using simple cross-validation.

Mixture of Networks for Clustering Categorical Data: A Penalized Composite Likelihood Approach

One of the challenges in clustering categorical data is the curse of dimensionality caused by the inherent sparsity of high-dimensional data, the records of which include a large number of attributes. The latent class model (LCM) assumes local independence between the variables in clusters, and is a parsimonious model-based clustering approach that has been used to circumvent the problem. The mixture of a log-linear model is more flexible but requires more parameters to be estimated. In this research, we recognize that each categorical observation can be conceived as a network with pairwise linked nodes, which are the response levels of the observation attributes. Therefore, the categorical data for clustering is considered a finite mixture of different component layer networks with distinct patterns. We apply a penalized composite likelihood approach to a finite mixture of networks for sparse multivariate categorical data to reduce the number of parameters, implement the EM algorithm to estimate the model parameters, and show that the estimates are consistent and satisfy asymptotic normality. The performance of the proposed approach is shown to be better in comparison with the conventional methods for both synthetic and real datasets.

Statistical Learning of Nonlinear Stochastic Differential Equations from Nonstationary Time Series using Variational Clustering

Parameter estimation for non-stationary stochastic differential equations (SDE) with an arbitrary non-linear drift and non-linear diffusion is accomplished in combination with a non-parametric clustering methodology. Such a model-based clustering approach includes a quadratic programming (QP) problem with equality and inequality constraints. We couple the QP problem to a closed-form likelihood function approach based on suitable Hermite-expansions to approximate the parameter values of the SDE model. The classification problem provides a smooth indicator function, which enables us to recover the underlying temporal parameter modulation of the one-dimensional SDE. As shown by the numerical examples, the clustering approach recovers a hidden functional relationship between the SDE model parameters and an additional auxiliary process. The study builds upon this functional relationship to develop closed-form, non-stationary, data-driven stochastic models for multiscale dynamical systems in real-world applications.

A model-based constrained deep learning clustering approach for spatially resolved single-cell data.

Spatially resolved scRNA-seq (sp-scRNA-seq) technologies provide the potential to comprehensively profile gene expression patterns in tissue context. However, the development of computational methods lags behind the advances in these technologies, which limits the fulfillment of their potential. In this study, we develop a deep learning approach for clustering sp-scRNA-seq data, named Deep Spatially constrained Single-cell Clustering (DSSC). In this model, we integrate the spatial information of cells into the clustering process in two steps: (1) the spatial information is encoded by using a graphical neural network model, and (2) cell-to-cell constraints are built based on the spatial expression pattern of the marker genes and added in the model to guide the clustering process. Then, a deep embedding clustering is performed on the bottleneck layer of autoencoder by Kullback-Leibler (KL) divergence along with the learning of feature representation. DSSC is the first model that can use information from both spatial coordinates and marker genes to guide cell/spot clustering. Extensive experiments on both simulated and real data sets show that DSSC boosts clustering performance significantly compared with the state-of-the-art methods. It has robust performance across different data sets with various cell type/tissue organization and/or cell type/tissue spatial dependency. We conclude that DSSC is a promising tool for clustering sp-scRNA-seq data.

Integrating herbivore assemblages and woody plant cover in an African savanna to reveal how herbivores respond to ecosystem management.

African savannas are experiencing anthropogenically-induced stressors that are accelerating the increase of woody vegetation cover. To combat this, land managers frequently implement large-scale clearing of trees, which can have a cascading influence on mammalian herbivores. Studies rarely focus on how differences in woody cover influence the herbivore assemblage, making it difficult to assess how aggressive measures, or the lack of management, to counteract increasing woody cover affect the local composition and biodiversity of herbivores. We address this knowledge gap by applying a model-based clustering approach to field observations from MalaMala Game Reserve, South Africa to identify multiple herbivore–vegetation ‘configurations,’ defined as unique sets of herbivore assemblages (i.e., groups of herbivores) associated with differing woody plant covers. Our approach delineated how tree-clearing influences the distribution and abundance of the herbivore community in relation to surrounding savanna areas, which represent a natural mosaic of varying woody cover. Regardless of season, both intensively managed areas cleared of trees and unmanaged areas with high tree cover contained configurations that had depauperate assemblages of herbivores (low species richness, low abundance). By contrast, habitats with intermediate cover of woody vegetation had much higher richness and abundance. These results have substantial implications for managing African savannas in a rapidly changing climate.

Bridging the Gap Between Qualitative and Quantitative Assessment in Science Education Research with Machine Learning — A Case for Pretrained Language Models-Based Clustering

Science education researchers typically face a trade-off between more quantitatively oriented confirmatory testing of hypotheses, or more qualitatively oriented exploration of novel hypotheses. More recently, open-ended, constructed response items were used to combine both approaches and advance assessment of complex science-related skills and competencies. For example, research in assessing science teachers’ noticing and attention to classroom events benefitted from more open-ended response formats because teachers can present their own accounts. Then, open-ended responses are typically analyzed with some form of content analysis. However, language is noisy, ambiguous, and unsegmented and thus open-ended, constructed responses are complex to analyze. Uncovering patterns in these responses would benefit from more principled and systematic analysis tools. Consequently, computer-based methods with the help of machine learning and natural language processing were argued to be promising means to enhance assessment of noticing skills with constructed response formats. In particular, pretrained language models recently advanced the study of linguistic phenomena and thus could well advance assessment of complex constructs through constructed response items. This study examines potentials and challenges of a pretrained language model-based clustering approach to assess preservice physics teachers’ attention to classroom events as elicited through open-ended written descriptions. It was examined to what extent the clustering approach could identify meaningful patterns in the constructed responses, and in what ways textual organization of the responses could be analyzed with the clusters. Preservice physics teachers (N = 75) were instructed to describe a standardized, video-recorded teaching situation in physics. The clustering approach was used to group related sentences. Results indicate that the pretrained language model-based clustering approach yields well-interpretable, specific, and robust clusters, which could be mapped to physics-specific and more general contents. Furthermore, the clusters facilitate advanced analysis of the textual organization of the constructed responses. Hence, we argue that machine learning and natural language processing provide science education researchers means to combine exploratory capabilities of qualitative research methods with the systematicity of quantitative methods.

Genomic stratification and differential natural selection signatures among human norovirus genogroup II isolates.

Noroviruses (NoVs), which are members of the family Caliciviridae, are the most common cause of gastroenteritis in humans. Ten NoV genogroups have been reported so far. Of these, genogroup II (GII) is the most prevalent, and it causes serious infections worldwide. The complete genome sequences of NoV GII isolates from different geographical regions were retrieved from the public database. The model-based clustering approach, implemented in the STRUCTURE resource, was employed for assessment of genetic composition. The MEGA X and IQ Tree tools were used for phylogenetic analysis. Genome-wide natural selection analysis was performed using maximum-likelihood-based methods. The demographic features of NoV GII genome sequences were assessed using the BEAST package. All of the NoV GII sequences initially clustered into two main subpopulations at significant K = 2, where the genotype GII.4 samples clearly split from the rest of the genotypes. This indicates a marked genetic distinction between norovirus GII.4 and non-GII.4 samples. Phylogenetic analysis showed the presence of five distinct subclades for genotype GII.2 and seven subclades for GII.4 samples. Several isolates with admixed ancestry were identified that constituted distinct subclusters in the phylogenetic tree. No continental-specific genetic distinctions were observed among the NoV GII samples. Significant genomic signatures of both positive and negative natural selection were identified across the NoV GII genes. A differential pattern of positive selection signals was inferred between the GII.4 and non-GII.4 genotypes. The demographic analysis revealed an increase in the effective population size of NoV GII during 2009-2010, followed by a rapid fall in 2015.Supplementary InformationThe online version contains supplementary material available at 10.1007/s00705-022-05396-9.

Assessment of genetic diversity of Moroccan Pistacia lentiscus L. populations using ISSR markers

Pistacia lentiscus L. is an indigenous species of considerable ecological and economic importance. An understanding of the patterns of genetic variation within and between Moroccan populations of this species is essential to design optimal genetic management strategies for its conservation. Here, inter simple sequence repeat (ISSR) technique was used to study genetic variation of 11 populations sampled from different regions in Morocco. The 13 primers used produced 121 bands which 110 bands (90.90 %) were polymorphic. The mean values of PIC, RP, I and Ht were 0.79; 4.89; 0.47 and 0.31 respectively, implying the important genetic variability between the studied populations. Moreover, AMOVA analysis showed that 58% of total genetic variability is accounted within population and 42% between populations. The UPGMA dendrogram and Bayesian model-based clustering approach identified five gene pools structured independently from the geographical origin of populations. These results indicate that the ISSRs markers used represent an efficient and important tool for the genetic analysis of the lentisk and the existence of a large genetic variability in this species in Morocco

Profiles and predictors of healthcare utilization: using a cluster-analytic approach to identify typical users across conventional, allied and complementary medicine, and self-care

IntroductionThe identification of typologies of health care users and their specific characteristics can be performed using cluster analysis. This statistical approach aggregates similar users based on their common health-related behavior. This study aims to examine health care utilization patterns using cluster analysis; and the associations of health care user types with sociodemographic, health-related and health-system related factors.MethodsCross-sectional data from the 2012 National Health Interview Survey were used. Health care utilization was measured by consultations with a variety of medical, allied and complementary health practitioners or the use of several interventions (exercise, diet, supplementation etc.) within the past 12 months (used vs. not used). A model-based clustering approach based on finite normal mixture modelling, and several indices of cluster fit were determined. Health care utilization within the cluster was analyzed descriptively, and independent predictors of belonging to the respective clusters were analyzed using logistic regression models including sociodemographic, health- and health insurance-related factors.ResultsNine distinct health care user types were identified, ranging from nearly non-use of health care modalities to over-utilization of medical, allied and complementary health care. Several sociodemographic and health-related characteristics were predictive of belonging to the respective health care user types, including age, gender, health status, education, income, ethnicity, and health care coverage.ConclusionsCluster analysis can be used to identify typical health care utilization patterns based on empirical data; and those typologies are related to a variety of sociodemographic and health-related characteristics. These findings on individual differences regarding health care access and utilization can inform future health care research and policy regarding how to improve accessibility of different medical approaches.

Bayesian nonparametric change point detection for multivariate time series with missing observations

A time series is a commonly observed type of data, and it is analyzed in several ways in real applications. Within the possible analysis, change point detection is one of the crucial inferential targets for studying the behavior of a time series. We consider a multiple change point detection model for a multivariate time series. Among the possible approaches to perform multiple change point detection, we propose an extension to the multivariate case of one of the main state-of-the-art approaches, working in a Bayesian nonparametric framework. We combine a combinatorial prior distribution, which relies on a model-based clustering approach to detect the change points, with a multivariate kernel for time-dependent realizations in a general fashion. We further extend the model to the case of missing observations and derive opportune quantities to perform data imputation. Thereafter, we investigate the properties of the multivariate model with an extensive simulation study, and we apply the model to perform change point detection in two real data applications.