Probabilistic Mixture Model Research Articles

Machine Learning Models Informed by Connected Mixture Components for Short- and Medium-Term Time Series Forecasting

This paper presents a new approach in the field of probability-informed machine learning (ML). It implies improving the results of ML algorithms and neural networks (NNs) by using probability models as a source of additional features in situations where it is impossible to increase the training datasets for various reasons. We introduce connected mixture components as a source of additional information that can be extracted from a mathematical model. These components are formed using probability mixture models and a special algorithm for merging parameters in the sliding window mode. This approach has been proven effective when applied to real-world time series data for short- and medium-term forecasting. In all cases, the models informed by the connected mixture components showed better results than those that did not use them, although different informed models may be effective for various datasets. The fundamental novelty of the research lies both in a new mathematical approach to informing ML models and in the demonstrated increase in forecasting accuracy in various applications. For geophysical spatiotemporal data, the decrease in Root Mean Square Error (RMSE) was up to 27.7%, and the reduction in Mean Absolute Percentage Error (MAPE) was up to 45.7% compared with ML models without probability informing. The best metrics values were obtained by an informed ensemble architecture that fuses the results of a Long Short-Term Memory (LSTM) network and a transformer. The Mean Squared Error (MSE) for the electricity transformer oil temperature from the ETDataset had improved by up to 10.0% compared with vanilla methods. The best MSE value was obtained by informed random forest. The introduced probability-informed approach allows us to outperform the results of both transformer NN architectures and classical statistical and machine learning methods.

Improving drought monitoring using climate models with bias‐corrected under Gaussian mixture probability models

Improving drought monitoring using climate models with bias‐corrected under Gaussian mixture probability models

Patterns of ultra-processed foods consumption throughout childhood and trajectories of growth and adiposity

Background & aimsUltra-processed foods (UPF) consumption has been associated with unhealthy outcomes. However, the literature lacks robust longitudinal studies considering its cumulative effect, particularly in young populations. This study aimed to evaluate the relationship between UPF consumption patterns throughout childhood with growth and adiposity trajectories. MethodsGeneration XXI population-based birth cohort (Porto, Portugal) participants were included. Food frequency questionnaire items at 4, 7 and 10 years were classified according to the processing degree using NOVA. UPF consumption patterns based on total quantity were identified using a probabilistic Gaussian mixture model using participants with complete data and predicting for the total sample (n=8647). To assess whether the outcome trajectories from 4 to 13 years [body weight (kg), height (cm), body mass index (BMI) z-score, waist circumference (WC) (cm) and fat mass (FM) (%)] depend on UPF patterns, a mixed-effects model with linear and quadratic terms for age adjusted for confounders was used. Participants with at least 2 measurements at 4, 7, 10 or 13 years were included in this study (n range: 5885-6272). ResultsFour UPF consumption patterns were identified: constantly lower consumption (15.4%), constantly intermediate consumption (56.4%), transition from low to high consumption (11.2%), and constantly higher consumption (17.1%). Compared to the constantly lower UPF consumption, the constantly higher consumption pattern was associated with greater acceleration in body weight (β: 0.119; 95%CI: 0.027;0.212), BMI z-score (β: 0.014; 95%CI: 0.004;0.023), WC (β: 0.232; 95%CI: 0.144;0.319) and FM% (β: 0.200; 95%CI: 0.092;0.308) and with lower acceleration in height (β: -0.063; 95%CI: -0.111;-0.015). The constantly intermediate UPF consumption pattern was associated with greater acceleration in body weight (β: 0.123; 95%CI: 0.043;0.203), WC (β: 0.120; 95%CI: 0.045;0.195) and FM% (β: 0.146; 95%CI: 0.054;0.238). ConclusionConstantly higher and constantly intermediate UPF consumption throughout childhood were associated with worse growth and adiposity trajectories until adolescence.

Association between calcium intake from different food sources during childhood and cardiometabolic risk on adolescence: The Generation XXI birth cohort.

Calcium intake has been associated with lower adiposity, but few studies explored the longitudinal relation of calcium from different sources and cardiometabolic markers in young population. Prospectively estimate the association between dairy and non-dairy calcium intake at 4, 7, and 10 years (y) of age and cardiometabolic risk at 13 y. The sample included 4017 participants from the Generation XXI birth cohort. Dietary data were collected from a 3-day food diary. Cardiometabolic clusters at 13 y were estimated by a probabilistic Gaussian mixture model (z-score of waist circumference [WC], HOMA-IR; HDL cholesterol, triglycerides, and systolic blood pressure [BP]). Multivariable linear and logistic regression models were used to estimate associations. Calcium intake (/100 mg), after adjustment for confounders, was negatively and significantly associated with body mass index (BMI) (β = -0.02, 95% CI: -0.04; -0.01), WC (cm) (β = -0.23, 95% CI: -0.36; -0.11), and diastolic BP (mmHg) (β = -0.14, 95% CI: -0.26; -0.03). After additional adjustment for total energy intake, associations lose statistical significance. Calcium intake from milk at 7 y was inversely associated with WC (β = -0.25, 95% CI: -0.48; -0.03) and from yogurt at 10 y was associated with higher BMI (β = 0.08, 95% CI: 0.03; 0.13) and WC (β = 0.54, 95% CI: 0.12; 0.96). Calcium from vegetables at 4, 7, 10 y reduces later cardiometabolic risk (OR = 0.71; OR = 0.84; OR = 0.98, respectively). This study supports a protective effect of calcium on adolescents' cardiometabolic health, especially from vegetables.

Unsupervised outlier detection using random subspace and subsampling ensembles of Dirichlet process mixtures

Probabilistic mixture models are recognized as effective tools for unsupervised outlier detection owing to their interpretability and global characteristics. Among these, Dirichlet process mixture models stand out as a strong alternative to conventional finite mixture models for both clustering and outlier detection tasks. Unlike finite mixture models, Dirichlet process mixtures are infinite mixture models that automatically determine the number of mixture components based on the data. Despite their advantages, the adoption of Dirichlet process mixture models for unsupervised outlier detection has been limited by challenges related to computational inefficiency and sensitivity to outliers in the construction of outlier detectors. Additionally, Dirichlet process Gaussian mixtures struggle to effectively model non-Gaussian data with discrete or binary features. To address these challenges, we propose a novel outlier detection method that utilizes ensembles of Dirichlet process Gaussian mixtures. This unsupervised algorithm employs random subspace and subsampling ensembles to ensure efficient computation and improve the robustness of the outlier detector. The ensemble approach further improves the suitability of the proposed method for detecting outliers in non-Gaussian data. Furthermore, our method uses variational inference for Dirichlet process mixtures, which ensures both efficient and rapid computation. Empirical analyses using benchmark datasets demonstrate that our method outperforms existing approaches in unsupervised outlier detection.

Bayesian peak identification method for low count rate gamma spectrum under short-time measurement based on physical priors

A Bayesian peak identification method based on physical model had been proposed in this study to perform qualitative analysis for the low count rate gamma spectrum under short-time measurement. Based on the prior knowledge about gamma detection, the distribution of energy deposited in the detector near the region of interest of a target peak through photoelectric effect, Compton scattering and multiple scattering had been analyzed and approximated using three elementary components, then, the distribution of each component after convolution had been studied, and at last, a probabilistic mixture model had been proposed to directly describe the distribution of measured energy within region of interest. When the data measurement has been completed, a Gibbs sampler was used to estimate the posterior distribution of model parameters to discriminate the existence of target peaks. The numerical and physical experiments were performed to verify the effectiveness of the parameter estimation algorithm and the performance of the proposed identification method. The numerical experiments showed the identification precision is proportional to counts and peak-to-total ratio, when the peak-to-total ratio is greater than 0.5, the proposed method could identify the peak with a probability greater than 50% given 50 observations. The physical experiment used a NaI(Tl) detector to measure 137Cs, 60Co and 152Eu sources in a laboratory environment at different equivalent distances, the peak identification precision had been tested for the proposed method, and the experiments proved that the probabilistic model is a good approximation to the physical process, and the proposed identification method outperformed the conventional method under short-time measurement conditions.

Goodness-of-fit, identifiability and extrapolation: Can the Two-Component Extreme Value distribution be used in at-site flood frequency analysis?

When fitting three-parameter flood frequency models to annual maximum (AM) flood series, the lack-of-fit can be mitigated by censoring potentially influential low flows (PILFs). An alternative and less-studied approach is to apply mixture probability models with four or more parameters, which trade off greater flexibility to fit AM series against the need to deal with degeneracy which arises when there is insufficient information to identify mixture components. However, the issue of degeneracy and the lack of a robust inference framework present a significant barrier to adoption. This study investigated the potential of the most parsimonious mixture model, the four-parameter Two-Component Extreme Value (TCEV) model. A Bayesian framework using Markov chain Monte Carlo sampling was developed to robustly characterize parameter uncertainty even in the presence of degeneracy. Two new posterior diagnostics based on the strength of the TCEV components were developed to aid identification of degeneracy. Armed with a robust inference framework, the study evaluated the potential of TCEV using a case study based on 31 catchments in eastern Australia with records exceeding 70 years. The evaluation used short to long records to compare TCEV and Log Pearson III fit and extrapolative uncertainty. The Log Pearson approach (LP3-PILF) censors PILFs following the approach described in Australian Rainfall and Runoff. With respect to goodness-of-fit, we found in most cases that TCEV fitted AM flood peaks well without the need to censor or stratify data, and overall, no clear difference emerged between TCEV and LP3-PILF. However, and contrary to expectation, TCEV produced high flow quantile confidence intervals consistently narrower than LP3-PILF even in the presence of degeneracy. While more case studies in different regions are needed to confirm the potential of TCEV, this study is a reminder that goodness-of-fit is a necessary but not sufficient criterion for selecting the probability model that best represents flood frequency.

Coherent point drift with Skewed Distribution for accurate point cloud registration

Point cloud registration methods based on Gaussian Mixture Models (GMMs) exhibit high robustness. However, GMM cannot precisely depict point clouds, because the Gaussian distribution is spatially symmetric and local surfaces of point clouds are typically non-symmetric. In this paper, we propose a novel method for rigid point cloud registration, termed coherent point drift with Skewed Distribution (Skewed CPD). Our method employs an asymmetric distribution constructed from the local surface normals and curvature radii. Compared to the Gaussian distribution, this skewed distribution provides a more accurate spatial description of points on local surfaces. Additionally, we integrate an adaptive multiplier to the covariance, which reallocates the weight of the covariance for different components in the probabilistic mixture model. We employ the EM algorithm to address this maximum likelihood estimation (MLE) issue and leverage GPU acceleration. In the M-step, we adopt an unconstrained optimization technique rooted in a Lie group and Lie algebra to attain the optimal transformation. Experimental results indicate that our method outperforms state-of-the-art methods in both accuracy and robustness. Remarkably, even without loop closure detection, the cumulative error of our approach remains minimal.

A Data-Driven Model with Hysteresis Compensation for I2RIS Robot.

Retinal microsurgery is a high-precision surgery performed on a delicate tissue requiring the skill of highly trained surgeons. Given the restricted range of instrument motion in the confined intraocular space, snake-like robots may prove to be a promising technology to provide surgeons with greater flexibility, dexterity, and positioning accuracy during retinal procedures such as retinal vein cannulation and epiretinal membrane peeling. Kinematics modeling of these robots is an essential step toward accurate position control. Unlike conventional manipulators, modeling these robots does not follow a straightforward method due to their complex mechanical structure and actuation mechanisms. The hysteresis problem can especially impact the positioning accuracy significantly in wire-driven snake-like robots. In this paper, we propose a data-driven kinematics model using a probabilistic Gaussian mixture model (GMM) and Gaussian mixture regression (GMR) approach with a hysteresis compensation algorithm. Experimental results on the two-degree-of-freedom (DOF) integrated robotic intraocular snake (I2RIS) show that the proposed model with the hysteresis compensation can predict the snake tip bending angle for pitch and yaw with 0.45° and 0.39° root mean square error (RMSE), respectively. This results in overall 60% and 70% improvements of accuracy for yaw and pitch over the same model without the hysteresis compensation.

Substructure heterogeneity during hot deformation of ferritic stainless steels - Experimental characterization and discussion assisted by a mean-field model

The microstructure evolution of ferritic stainless steels during hot deformation is complex and needs to be finely described. Quantifying the evolution of the misorientation of low-angle boundaries depending on the thermomechanical path is a key point in controlling the recrystallization of such steels. This paper proposes an experimental methodology based on large-scale electron back-scattered diffraction (EBSD) mapping characterization using a Symmetry 2 camera to track the low-angle boundary evolution. Different thermomechanical paths, from 900 to 1100 °C with strain from 0.3 to 0.9, were studied using uniaxial compression tests. Flow stress follows a usual hardening stage, followed by a slight softening regime, mostly attributed to continuous dynamic recrystallization. Distributions of low-angle boundary populations (density vs sub-grain size) are quantified by two populations either near grain boundaries or inside grain bulk; the proportion of sub-grains at grain boundaries increases with strain. This bimodal distribution is made with a probabilistic Gaussian mixture model. These results are discussed using a modified mean-field model of Gourdet-Montheillet. This model is insufficient to capture the transient stage of continuous dynamic recrystallization unless its set of parameters is adjusted to accommodate the saturation of the density of low-angle boundaries near grain boundaries. This saturation is a result of the low-angle boundary density gradients in the grains.

Learned relevance of a distracting cue can influence feature interference errors

ABSTRACT Previous research has shown that attention can be guided via past experiences and learned regularities of target and/or distractor location. Here, we ask: could the learned relevance of a distracting cue affect our ability to suppress it and prevent feature interference errors? Participants reported the colour of a target item, while a salient distractor cue appeared on some trials. Critically, the experiment was split into two contexts, which differed based on whether the target and distracting cue could appear at the same location. We used a probabilistic mixture model to estimate feature report errors. We found a significant difference in swap rates (misreporting the colour at the salient distractor location instead of target colour) between the two contexts depending on the order they were experienced. These results suggest that the learned relevance of a distractor cue can affect how likely it is to capture attention and impact target feature perception.

Bisphenol A and cardiometabolic risk in adolescents: Data from the Generation XXI cohort

Background and aimsBisphenol A (BPA), an endocrine disruptor widely used in food contact materials, has been linked to a worse health profile. This study intends to estimate the association between BPA exposure and cardiometabolic patterns at adolescence. Methods and resultsData from the Portuguese population-based birth cohort Generation XXI at the age of 13 were used (n = 2386 providing 3-day food diaries and fasting blood samples). BPA exposure was measured in 24-h urine from a subsample (n = 206) and then predicted in all participants using a random forest method and considering dietary intake from diaries. Three cardiometabolic patterns were identified (normal, modified lipid profile and higher cardiometabolic risk) using a probabilistic Gaussian mixture model. Multinomial regression models were applied to associate BPA exposure (lower, medium, higher) and cardiometabolic patterns, adjusting for confounders. The median BPA exposure was 1532 ng/d, corresponding to 29.4 ng/kg/d. Adolescents higher exposed to BPA (compared to medium and lower levels) had higher BMI z-score (kg/m2) (0.68 vs. 0.39 and 0.52, respectively; p = 0.008), higher levels of body fat (kg) (16.3 vs. 13.8 and 14.6, respectively; p = 0.002), waist circumference (76.2 vs. 73.7 and 74.9, respectively; p = 0.026), insulinemia (ug/mL) (14.1 vs. 12.7 and 13.1, respectively; p = 0.039) and triglyceridemia (mg/dL) (72.7 vs. 66.1 and 66.5, respectively; p = 0.030). After adjustment, a significant association between higher BPA and a higher cardiometabolic risk pattern was observed (OR: 2.55; 95%CI: 1.41, 4.63). ConclusionHigher BPA exposure was associated with a higher cardiometabolic risk pattern in adolescents, evidencing the role of food contaminants in health.

Unsupervised Linear Component Analysis for a Class of Probability Mixture Models

Unsupervised Linear Component Analysis for a Class of Probability Mixture Models

ANFIS learning using expectation maximization based Gaussian mixture model and multilayer perceptron learning

The Adaptive Neuro-Fuzzy Inference System (ANFIS) is a hybrid learning algorithm that combines the learning ability of neural networks with fuzzy inference systems. While ANFIS has been successfully applied to several real-world problems, effective parameter optimization remains a challenge. This paper presents a novel approach for optimizing parameters of ANFIS in supervised settings. The proposed approach utilizes a combination of probabilistic mixture models and perceptron-based learning to parameterize ANFIS. To learn ANFIS membership functions, it generates a mixture of the finite probability distributions for each input feature. Then the consequent parameters are learned by transforming them into weights of a multi-layer perceptron instance. The effectiveness of the proposed method is evaluated on classification problems of varying complexity, ranging from binary-class to multi-class with variable dimensions. The results show that the proposed algorithm improves ANFIS performance in terms of accuracy rate and speed (train-time) analysis. The effectiveness and efficiency of the proposed approach have been further confirmed using a 5 × 2 cross-validation paired significance t-test. In particular, for binary-class problems, our model outperformed standard methods by up to 10% accuracy improvement. Similarly, for multi-class problems, our model achieved an average increase of 8% in accuracy while reaching up to 14% improvement. On average, the proposed model showed a reduction of about 75% in training time compared to the other models. Overall, the proposed approach offers competitive computational performance and acceptable efficacy for parameter optimization of ANFIS.

Ignition Delay Time Modeling in Wire-EDM

This study presents a comprehensive investigation and modeling of the ignition delay time (td) in wire-EDM (WEDM). The research focuses on the influence of gap distance, discharge energy, and piece height on the stochastic distributions of td, providing important insights into the complex properties of these distributions. Observations indicate that these parameters exert significant yet intricate influences on td, with a particular emphasis on the gap distance. A critical value was identified, around 8μm to 10μm, that divides the stochastic behavior. To capture the binomial nature of td, a mixture probability model consisting of two Weibull distribution curves was developed and validated through extensive experimentation and a data analysis. The model demonstrated strong agreement with observed cumulative probability curves, indicating its accuracy and reliability in predicting td. Further, a sensitivity analysis revealed regions of fast change, emphasizing the challenges and importance of careful parameter selection in control of WEDM processes. The findings of this study contribute to a deeper understanding of WEDM processes and provide a modeling approach for predicting td. Future research directions include refining the model by incorporating additional input parameters, investigating the influence of other process variables on td.

Probabilistic model of traffic scenarios for extreme load effects in long-span bridges

The traffic scenarios that may cause extreme load effects are of great importance to the safety assessment of bridge structures. The traditional simulation method of traffic flow cannot depict the distribution pattern of vehicles on the bridge deck when the maximum effect is induced. In this paper, a probabilistic Gaussian mixture model (GMM) for heavy vehicle scenarios on the bridge deck under free-flow condition is proposed for long-span bridges based on collected Weigh in Motion (WIM) data. The scenarios of extreme response under free-flow occur more frequently than congestion scenarios and are of similar value and relevance in the daily management and safety assessment of long-span bridges.A non-stationary Poisson process is utilized to simulate the uneven occurrence of heavy vehicles in different lanes, and it is assumed that they are located within the artificially defined cells on the bridge deck. Then, Nataf transformation is employed to consider the correlation of gross vehicle weights (GVWs) within close range in the same lane. The numerical study is carried out on a long-span cable-stayed bridge to investigate the effects of correlation in GVWs and stationarity of vehicle distribution location on the structural responses. The load responses calculated by the proposed model and Monte Carlo method for different effects are compared with the values derived from code model. The results show that with the increase of the correlation level of the neighboring GVWs, the simulated responses are more prone to get extreme values, which means an increasing probability of the most unfavorable spatial distribution of on-bridge vehicles. The same results are also found under the non-stationary simulation state for vehicle location. The non-stationary Poisson process provides an efficient, highly feasible method, which is also in the safe side, for simulating the vehicle spatial distribution for specific effects.

Hybrid Message Passing Channel Estimation Algorithm for Massive MIMO-OFDM Systems

In this letter, we propose the Bernoulli two-state Gaussian mixture (B-TSGM) probability model to characterize the angle-delay domain (ADD) channel of the massive multiple-input multiple-output orthogonal frequency-division multiplexing (MIMO-OFDM) system. Based on the hybrid message passing (HMP) rule, we design the HMP-B-TSGM channel estimation algorithm under the structured turbo-compressed sensing (STCS) framework. Simulation results show that the considered model better captures the channel characteristics, and the proposed algorithm outperforms the state-of-the-art methods under a wide range of simulation settings while having the same complexity.

A discrete choice modeling framework of heterogenous decision rules accounting for non-trading behavior

We present a discrete choice modeling framework with heterogeneous decision rules accounting for non-trading behavior. The proposed approach builds upon the state-of-the-art probabilistic finite mixture models and tackles non-trading behavior while accounting for inertia effects and serial correlation in the SP data, and contextual effects on the probability of an individual employing a specific decision rule. The framework involves three subpopulations of decision-makers, referred to respectively as pure utility-maximizers, utility-maximizers with strong preference for one alternative, and non-traders non-utility-maximizers employing a non-trading heuristic. The second subpopulation is expected to exhibit non-trading behavior, despite making trade-offs consistent with utility maximization. Our goal is to disentangle the two types of manifested non-trading behavior. We assume that the manifestation of non-trading behavior – by otherwise utility-maximizing individuals – may be driven by important context variables. In order to accommodate this assumption in the modeling framework, we define and add a relative advantage (RA) component in the class-membership model. Finally, we apply the framework to a Swiss stated preferences (SP) mode choice case study, and demonstrate the impact of accounting for non-trading behavior on the value of time estimates.

Mixture Probability Models with Covariates: Applications in Estimating Risk of Hydroclimatic Extremes

Modeling of extreme events is important in many scientific fields, including environmental and civil engineering, and impacts and risk assessments. Among available methods, statistical models that allow estimating extremes’ frequency and intensity are regularly used in procedures to anticipate potential changes in extreme events. Extreme value theory provides a theoretical basis for statistical estimation of extreme events using frequency analysis. The challenge in modeling is knowing when to use the block maxima method or the peaks-over-threshold (POT) method. Each has its drawbacks. POT describes the main characteristics of the observed extreme series; the threshold selection is always challenging and might affect the accuracy of the simulated results and the credibility of changes in extreme values. To encompass this challenge, mixture models offer more flexibility to represent samples with nonhomogeneous data. This study presents the gamma generalized Pareto (GGP) mixture model for estimating risk occurrence of hydroclimatic extremes. The model was developed in its general form, whereas the observed hydrometeorological extreme events depend on multidimensional covariates. A maximum likelihood algorithm is proposed to estimate the parameters with a constraint on the shape parameter of the generalized Pareto (GP) distribution. A Monte Carlo (MC) simulation compared the proposed model with the classical POT approach, with a fixed threshold, and the annual maximum series of streamflow. The approach was applied using a daily hydrological data set from an observed station located in the Saint John River at Fort Kent (01AD002), New Brunswick, Canada. The results show a flexibility to model extremes for dependent or nonstationary time series and adequately describes the central part of the observed frequencies, as well as the tails.

Probabilistic Mixture Models Improve Calibration of Panel-derived Tumor Mutational Burden in the Context of both Tumor-normal and Tumor-only Sequencing.

Tumor mutational burden (TMB) has been investigated as a biomarker for immune checkpoint blockade (ICB) therapy. Increasingly, TMB is being estimated with gene panel-based assays (as opposed to full exome sequencing) and different gene panels cover overlapping but distinct genomic coordinates, making comparisons across panels difficult. Previous studies have suggested that standardization and calibration to exome-derived TMB be done for each panel to ensure comparability. With TMB cutoffs being developed from panel-based assays, there is a need to understand how to properly estimate exomic TMB values from different panel-based assays. Our approach to calibration of panel-derived TMB to exomic TMB proposes the use of probabilistic mixture models that allow for nonlinear relationships along with heteroscedastic error. We examined various inputs including nonsynonymous, synonymous, and hotspot counts along with genetic ancestry. Using The Cancer Genome Atlas cohort, we generated a tumor-only version of the panel-restricted data by reintroducing private germline variants. We were able to model more accurately the distribution of both tumor-normal and tumor-only data using the proposed probabilistic mixture models as compared with linear regression. Applying a model trained on tumor-normal data to tumor-only input results in biased TMB predictions. Including synonymous mutations resulted in better regression metrics across both data types, but ultimately a model able to dynamically weight the various input mutation types exhibited optimal performance. Including genetic ancestry improved model performance only in the context of tumor-only data, wherein private germline variants are observed. A probabilistic mixture model better models the nonlinearity and heteroscedasticity of the data as compared with linear regression. Tumor-only panel data are needed to properly calibrate tumor-only panels to exomic TMB. Leveraging the uncertainty of point estimates from these models better informs cohort stratification in terms of TMB.