Dirichlet Mixture Model Research Articles

Identification of topical issues of the energy transition in publications of the scientific content aggregator Scilit. Part 2. Keyword clustering

Background. The access of Russian researchers to Scopus and Web of Science has become restricted, so the use of open reference databases becomes relevant. Objective. Identification of topical problems of energy transition in publications presented in Scilit, a content aggregator for scientific publications with free access. Materials and methods. The study utilized 10,121 bibliometric records of articles from 2019–2023. Publications were systematized using Gibbs sampling algorithm for Dirichlet mixture model. The topics of publications within the obtained clusters were analyzed using the demo version of the Carrot2 program. Publications were ranked using the sumy utility with the lex-rank algorithm. Results. The identified topical topics are devoted to systemic problems of energy complexes, including integration of different sources of energy generation, energy storage in “accumulators” or “green hydrogen” and optimization of their operation. Much attention is paid to the social aspects of the energy transition, especially relevant for rural areas and regions with a low level of economic development. Conclusions. Without financial support and appropriate infrastructure for local energy communities, the energy transition may be rejected by them. Households should be encouraged to use cleaner energy sources that are less harmful to health and the environment.

Bayesian clustering of spatially distributed compositional data with application to the Great Barrier Reef.

The relative abundance of groups of species is often used in ecological surveys to estimate community composition, a metric that reflects patterns of commonness and rarity of biological assemblages. The focus of this paper is measurements of the abundances of four benthic groups (that live on the seafloor) at several reefs on Australia's Great Barrier Reef (GBR) gathered between 2012 and 2017. In this paper we develop a statistical model to find clusters of locations with similar composition. We examine the changes in clusters during a period impacted by an unprecedented sequence of extreme environmental disturbances. To achieve this, we propose a model that incorporates the geographical location of the data, accounting for the possibility that nearby reefs are similar in composition. This is accomplished with a Dirichlet mixture model and a Potts distribution on the cluster assignments. Non-availability of the normalised Potts distribution makes Bayesian inference a doubly-intractable task. To circumvent this additional inferential challenge, an approximate exchange algorithm is specified. The analysis of the 2012 data, collected before the weather disturbances, reveals four clusters. The four groups highlight the primary habitat patterns in the 2012 GBR, each with distinct ecological characteristics: (1) areas with above-average soft coral abundance, (2) sand-dominated regions commonly found in the central part, (3) southern reefs with a more balanced distribution of species, and (4) habitats dominated by algae and hard corals. Compared to subsequent surveys conducted after disturbances, there is evidence of a decline in the number of clusters and a simplification of reef composition at the regional scale.

A Lightweight Intrusion Detection System Using a Finite Dirichlet Mixture Model With Extended Stochastic Variational Inference

A Lightweight Intrusion Detection System Using a Finite Dirichlet Mixture Model With Extended Stochastic Variational Inference

Hierarchical mixture of discriminative Generalized Dirichlet classifiers

This paper presents a discriminative classifier for compositional data. This classifier is based on the posterior distribution of the Generalized Dirichlet which is the discriminative counterpart of Generalized Dirichlet mixture model. Moreover, following the mixture of experts paradigm, we proposed a hierarchical mixture of this classifier. In order to learn the models parameters, we use a variational approximation by deriving an upper-bound for the Generalized Dirichlet mixture. To the best of our knownledge, this is the first time this bound is proposed in the literature. Experimental results are presented for spam detection and color space identification.

Covering Hierarchical Dirichlet Mixture Models on binary data to enhance genomic stratifications in onco-hematology.

Onco-hematological studies are increasingly adopting statistical mixture models to support the advancement of the genomically-driven classification systems for blood cancer. Targeting enhanced patients stratification based on the sole role of molecular biology attracted much interest and contributes to bring personalized medicine closer to reality. In onco-hematology, Hierarchical Dirichlet Mixture Models (HDMM) have become one of the preferred method to cluster the genomics data, that include the presence or absence of gene mutations and cytogenetics anomalies, into components. This work unfolds the standard workflow used in onco-hematology to improve patient stratification and proposes alternative approaches to characterize the components and to assign patient to them, as they are crucial tasks usually supported by a priori clinical knowledge. We propose (a) to compute the parameters of the multinomial components of the HDMM or (b) to estimate the parameters of the HDMM components as if they were Multivariate Fisher's Non-Central Hypergeometric (MFNCH) distributions. Then, our approach to perform patients assignments to the HDMM components is designed to essentially determine for each patient its most likely component. We show on simulated data that the patients assignment using the MFNCH-based approach can be superior, if not comparable, to using the multinomial-based approach. Lastly, we illustrate on real Acute Myeloid Leukemia data how the utilization of MFNCH-based approach emerges as a good trade-off between the rigorous multinomial-based characterization of the HDMM components and the common refinement of them based on a priori clinical knowledge.

A Big Data Analytical Framework for Intrusion Detection Based On Novel Elephant Herding Optimized Finite Dirichlet Mixture Models

For the purpose of identifying a wide variety of hostile activity in cyberspace, an Intrusion Detection System (IDS) is a crucial instrument. However, traditional IDSs have limitations in detecting zero-day attacks, which can lead to high false alarm rates. To address this issue, it is crucial to integrate the monitoring and analysis of network data with decision-making methods that can identify anomalous events accurately. By combining these approaches, organizations can develop more effective cybersecurity measures and better protect their networks from cyber threats. In this study, we proposed a novel called the Elephant Herding Optimized Finite Dirichlet Mixture Model (EHO-FDMM). This framework consists of three modules: capture and logging, pre-processing, and an innovative IDS method based on the EHO-FDMM. The NSL-KDD and UNSW-NB15 datasets are used to assess this framework's performance. The empirical findings show that selecting the optimum model that accurately fits the network data is aided by statistical analysis of the data. The EHO-FDMM-based intrusion detection method also offers a lower False Alarm Rate (FPR) and greater Detection Rate (DR) than the other three strong methods. The EHO-FDMM and exact interval of confidence bounds were used to create the suggested method's ability to detect even minute variations between legal and attack routes. These methods are based on correlations and proximity measurements, which are ineffective against contemporary assaults that imitate everyday actions.

Integrated mixture model and ensemble learning geographic object-based image analysis for road network extraction

ABSTRACT Delineation of road networks is often hindered by shadows, occlusions and spectrally similar objects. A hybrid Geographic object-based Image analysis (GeOBIA) technique that combines Mixture Model segmentation with Ensemble Learning algorithms to extract road networks is proposed. The Multispectral Local Dirichlet Mixture Model (MLDMM) highlights the built-up area. The bagging or subspace integrated ensemble learning-based classification makes the framework immune to overfitting and a novel statistical feature selection phase boosts the performance by 18%. Post-processing by path morphology achieves complete networks. MLDMM-SDEC proffers a precision of 99.86%, a recall of 90.90%, an F1-score of 95.23%, a detection quality of 94.73%, and an accuracy of 96.77%.

Wind Speed Forecasting with a Clustering-Based Deep Learning Model

The predictability of wind energy is crucial due to the uncertain and intermittent features of wind energy. This study proposes wind speed forecasting models, which employ time series clustering approaches and deep learning methods. The deep learning (LSTM) model utilizes the preprocessed data as input and returns data features. The Dirichlet mixture model and dynamic time-warping method cluster the time-series data features and then deep learning in forecasting. Particularly, the Dirichlet mixture model and dynamic warping method cluster the time-series data features. Next, the deep learning models use the entire (global) and clustered (local) data to capture the long-term and short-term patterns, respectively. Furthermore, an ensemble model is obtained by integrating the global model and local model results to exploit the advantages of both models. Our models are tested on four different wind data obtained from locations in Turkey with different wind regimes and geographical aspects. The numerical results indicate that the proposed ensemble models achieve the best accuracy compared to the deep learning method (LSTM). The results imply that the feature clustering approach accommodates a promising framework in forecasting.

An enriched mixture model for functional clustering

AbstractThere is an increasingly rich literature about Bayesian nonparametric models for clustering functional observations. Most recent proposals rely on infinite‐dimensional characterizations that might lead to overly complex cluster solutions. In addition, while prior knowledge about the functional shapes is typically available, its practical exploitation might be a difficult modeling task. Motivated by an application in e‐commerce, we propose a novel enriched Dirichlet mixture model for functional data. Our proposal accommodates the incorporation of functional constraints while bounding the model complexity. We characterize the prior process through a urn scheme to clarify the underlying partition mechanism. These features lead to a very interpretable clustering method compared to available techniques. Moreover, we employ a variational Bayes approximation for tractable posterior inference to overcome computational bottlenecks.

Dirichlet Process Mixture of Generalized Inverted Dirichlet Distributions for Positive Vector Data With Extended Variational Inference.

A Bayesian nonparametric approach for estimation of a Dirichlet process (DP) mixture of generalized inverted Dirichlet distributions [i.e., an infinite generalized inverted Dirichlet mixture model (InGIDMM)] has been proposed. The generalized inverted Dirichlet distribution has been proven to be efficient in modeling the vectors that contain only positive elements. Under the classical variational inference (VI) framework, the key challenge in the Bayesian estimation of InGIDMM is that the expectation of the joint distribution of data and variables cannot be explicitly calculated. Therefore, numerical methods are usually applied to simulate the optimal posterior distributions. With the recently proposed extended VI (EVI) framework, we introduce lower bound approximations to the original variational objective function in the VI framework such that an analytically tractable solution can be derived. Hence, the problem in numerical simulation has been overcome. By applying the DP mixture technique, an InGIDMM can automatically determine the number of mixture components from the observed data. Moreover, the DP mixture model with an infinite number of mixture components also avoids the problems of underfitting and overfitting. The performance of the proposed approach is demonstrated with both synthesized data and real-life data applications.

GEOSCENE MODELING AND ANALYSIS FOR URBAN FUNCTIONAL ZONES

Abstract. Urban functional zones are basic units for human socio-economic activities in cities. Urban functional zones play an important role in a wide range of urban studies, as they are fundamental to urban planning and management, urban resource allocation, as well as urban sustainable development evaluation. However, existing studies of urban functional zone modeling and mapping mainly employ state-of-the-art image segmentation and deep learning classification methods in computer science field to delineate and recognize urban functional zones, but ignore their heterogeneity, integration, multi-scale, and locality characteristics, so that the accuracy of urban functional zone modeling, mapping, and analysis is limited and interpretable, thus cannot comprehensively meet the requirements of downstream applications. This study, however, proposes a geoscene modeling and analysis framework for extracting urban functional zones, which first delineates urban functional zones by multiscale geoscene segmentation, then characterizes them by hierarchical semantic cognition, and finally recognizes their functional categories by linear Dirichlet mixture model. Although this paper does not present the detailed implementations of the geoscene modeling of urban functional zones, it illustrates the importance and the generally strategy of geoscene modeling in urban functional zone mapping and analysis. Additionally, urban functional zone mapping results of Baoshan District, Shanghai, in 2015 and 2020 are presented to verify the effectiveness of the proposed geoscene modeling strategy.

A probabilistic multi-criteria evaluation framework for integrated energy system planning

For the planning of integrated energy system (IES), it is of great significance to accurately evaluate the performance of the system’s energy supply in a number of aspects including e.g., economy, operational efficiency, environmental effects and security. In order to address the difficulties associated with uncertainties of evaluation indices, one probabilistic multi-criteria evaluation (PMCE) framework is proposed in this study for a class of integrated energy supply systems. In this framework, the evaluation is carried out based on probability distributions instead of deterministic values of evaluation indices, and to determine the criterion weights in PMCE, a Dirichlet mixture model (DMM) is applied to determine the weights required in PMCE to represent the priorities of different criteria. A case study has been carried out to evaluate the overall performce of five energy supply systems with a smart community demonstrative project. The results show that the proposed approach outperforms existing average-based and maximum entropy principle (MEP)-based approaches and demonstrate good robustness when encountering changes in the prices of electricity and natural gas.

Stratification of the Gut Microbiota Composition Landscape across the Alzheimer's Disease Continuum in a Turkish Cohort.

ABSTRACTAlzheimer's disease (AD) is a heterogeneous disorder that spans a continuum with multiple phases, including preclinical, mild cognitive impairment, and dementia. Unlike for most other chronic diseases, human studies reporting on AD gut microbiota in the literature are very limited. With the scarcity of approved drugs for AD therapies, the rational and precise modulation of gut microbiota composition using diet and other tools is a promising approach to the management of AD. Such an approach could be personalized if an AD continuum can first be deconstructed into multiple strata based on specific microbiota features by using single or multiomics techniques. However, stratification of AD gut microbiota has not been systematically investigated before, leaving an important research gap for gut microbiota-based therapeutic approaches. Here, we analyze 16S rRNA amplicon sequencing of stool samples from 27 patients with mild cognitive impairment, 47 patients with AD, and 51 nondemented control subjects by using tools compatible with the compositional nature of microbiota. To stratify the AD gut microbiota community, we applied four machine learning techniques, including partitioning around the medoid clustering and fitting a probabilistic Dirichlet mixture model, the latent Dirichlet allocation model, and we performed topological data analysis for population-scale microbiome stratification based on the Mapper algorithm. These four distinct techniques all converge on Prevotella and Bacteroides stratification of the gut microbiota across the AD continuum, while some methods provided fine-scale resolution in stratifying the community landscape. Finally, we demonstrate that the signature taxa and neuropsychometric parameters together robustly classify the groups. Our results provide a framework for precision nutrition approaches aiming to modulate the AD gut microbiota.IMPORTANCE The prevalence of AD worldwide is estimated to reach 131 million by 2050. Most disease-modifying treatments and drug trials have failed, due partly to the heterogeneous and complex nature of the disease. Recent studies demonstrated that gut dybiosis can influence normal brain function through the so-called “gut-brain axis.” Modulation of the gut microbiota, therefore, has drawn strong interest in the clinic in the management of the disease. However, there is unmet need for microbiota-informed stratification of AD clinical cohorts for intervention studies aiming to modulate the gut microbiota. Our study fills in this gap and draws attention to the need for microbiota stratification as the first step for microbiota-based therapy. We demonstrate that while Prevotella and Bacteroides clusters are the consensus partitions, the newly developed probabilistic methods can provide fine-scale resolution in partitioning the AD gut microbiome landscape.

A Hybrid of Interactive Learning and Predictive Modeling for Occupancy Estimation in Smart Buildings

This paper proposes a statistical learning approach for estimating occupancy in smart buildings using a set of small and simple nonintrusive sensors that can be viewed as alternatives to sensors that are sometimes perceived as invasive such as cameras. In that context, large amount of labelled training data are required. However, labelling large scale occupancy data is time consuming and tedious since it requires the direct involvement of the users. To tackle this challenge, we consider a hybrid approach based on the recently introduced interactive learning methodology that allows to collect training data of good quality, by ensuring a minimal involvement of the user, and a classification approach that we have developed. The classification part is based on the predictive distribution of the generalized Dirichlet (GD) mixture model which unfortunately does not have a closed-form. To alleviate that issue, we calculate a reliable approximation to the predictive distribution by optimizing the parameters of GD posterior distribution by a Bayesian variational inference approach. The choice of the GD mixture model is motivated by the heterogeneous non-Gaussian nature of the sensors outputs. Extensive experimental results reported for both synthetic data and real data indicate that our method could achieve promising results especially with extremely small training data.

Useful life prediction based on wavelet packet decomposition and two-dimensional convolutional neural network for lithium-ion batteries

It is important to know the replace time for reducing the lithium-ion battery risk and assessing its reliability. For this purpose, the remaining useful life (RUL) can play an important role in the prognostics and health management of battery to solve the inaccurate prediction issue. The existing RUL prediction techniques for lithium-ion batteries are inefficient for learning long-term dependencies among capacity degradations. In this work, a new forecasting approach is proposed based on wavelet packet decomposition, two-dimensional convolutional neural network, and adaptive multiple error corrections. In this model, the bivariate Dirichlet mixture model is considered to make the heteroscedasticity of the unpredictable residuals signal based non-parametric distribution. To show the validity of the proposed model, the experimental data are considered based on Continental Europe and NASA Ames Prognostics Center of Excellence battery datasets. The obtained numerical analysis presents an accurate forecasting model. Different comparisons with the well-known models are made to show the validity of the suggested approach, which proves the superiority and forecasting stability of the proposed model.

Variational Bayesian Learning of SMoGs: Modelling and Their Application to Synthetic Aperture Radar

We show how modern Bayesian Machine Learning tools can be effectively used in order to develop efficient methods for filtering Earth Observation signals. Bayesian statistical methods can be thought of as a generalization of the classical least-squares adjustment methods where both the unknown signals and the parameters are endowed with probability distributions, the priors. Statistical inference under this scheme is the derivation of posterior distributions, that is, distributions of the unknowns after the model has seen the data. Least squares can then be thought of as a special case that uses Gaussian likelihoods, or error statistics. In principle, for most non-trivial models, this framework requires performing integration in high-dimensional spaces. Variational methods are effective tools for approximate inference in Statistical Machine Learning and Computational Statistics. In this paper, after introducing the general variational Bayesian learning method, we apply it to the modelling and implementation of sparse mixtures of Gaussians (SMoG) models, intended to be used as adaptive priors for the efficient representation of sparse signals in applications such as wavelet-type analysis. Wavelet decomposition methods have been very successful in denoising real-world, non-stationary signals that may also contain discontinuities. For this purpose we construct a constrained hierarchical Bayesian model capturing the salient characteristics of such sets of decomposition coefficients. We express our model as a Dirichlet mixture model. We then show how variational ideas can be used to derive efficient methods for bypassing the need for integration: the task of integration becomes one of optimization. We apply our SMoG implementation to the problem of denoising of Synthetic Aperture Radar images, inherently affected by speckle noise, and show that it achieves improved performance compared to established methods, both in terms of speckle reduction and image feature preservation.

Combined Improved Dirichlet Models and Deep Learning Models for Road Extraction from Remote Sensing Images

Combining Dirichlet Mixture Models (DMM) with deep learning models for road extraction is an attractive study topic. Benefiting from DMM, the manually labeling work is alleviated. However, DMM suffers from high computational complexity due to pixel by pixel computations. Also, traditional constant parameter settings of DMM may not be suitable for different target images. To address the above problems, we propose an improved DMM which embeds superpixel strategy and sparse representation into DMM. In our road extraction framework, we first use improved DMM to filter out most backgrounds. Then, a trained deep CNN model is used for further precise road area recognition. To further promote the processing speed, we also apply the superpixel scanning strategy for CNN models. We tested our method on a Shaoshan dataset and proved that our method not only can achieve better results than other compared state-of-the-art image segmentation methods, but the processing speed and accuracy of DMM are also improved.

Discriminative Learning Approach Based on Flexible Mixture Model for Medical Data Categorization and Recognition.

In this paper, we propose a novel hybrid discriminative learning approach based on shifted-scaled Dirichlet mixture model (SSDMM) and Support Vector Machines (SVMs) to address some challenging problems of medical data categorization and recognition. The main goal is to capture accurately the intrinsic nature of biomedical images by considering the desirable properties of both generative and discriminative models. To achieve this objective, we propose to derive new data-based SVM kernels generated from the developed mixture model SSDMM. The proposed approach includes the following steps: the extraction of robust local descriptors, the learning of the developed mixture model via the expectation–maximization (EM) algorithm, and finally the building of three SVM kernels for data categorization and classification. The potential of the implemented framework is illustrated through two challenging problems that concern the categorization of retinal images into normal or diabetic cases and the recognition of lung diseases in chest X-rays (CXR) images. The obtained results demonstrate the merits of our hybrid approach as compared to other methods.

Nonparametric learning approach based on infinite flexible mixture model and its application to medical data analysis

AbstractThe goal of this paper is to develop an effective approach allowing to capture accurately the intrinsic nature of data using an infinite shifted‐scaled Dirichlet mixture model (InSSDMM). This article extends the finite statistical model to a more efficient multidimensional infinite mixture. The flexibility of the developed framework is demonstrated via some challenging medical applications that concern diabetic retinopathy detection in eye images and pneumonia identification in chest X‐ray scans. The obtained results demonstrate the merits of such approach as compared to many other generative and discriminative models.

Unsupervised Learning Using Variational Inference on Finite Inverted Dirichlet Mixture Models with Component Splitting

Unsupervised learning has been one of the essentials of pattern recognition and data mining. The role of Dirichlet family of mixture models in this field is inevitable. In this article, we propose a finite Inverted Dirichlet mixture model for unsupervised learning using variational inference. In particular, we develop an incremental algorithm with a component splitting approach for local model selection, which makes the clustering algorithm more efficient. We illustrate our model and learning algorithm with synthetic data and some real applications for occupancy estimation in smart homes and topic learning in images and videos. Extensive comparisons with comparable recent approaches have shown the merits of our proposed model.