Hierarchical Probabilistic Model Research Articles

Development of a probabilistic risk model for pharmaceuticals in the environment under population and wastewater treatment scenarios.

Preparing for future environmental pressures requires projections of how relevant risks will change over time. Current regulatory models of environmental risk assessment (ERA) of pollutants such as pharmaceuticals could be improved by considering the influence of global change factors (e.g., population growth) and by presenting uncertainty more transparently. In this article, we present the development of a prototype object-oriented Bayesian network (BN) for the prediction of environmental risk for six high-priority pharmaceuticals across 36 scenarios: current and three future population scenarios, combined with infrastructure scenarios, in three Norwegian counties. We compare the risk, characterized by probability distributions of risk quotients (RQs), across scenarios and pharmaceuticals. Our results suggest that RQs would be greatest in rural counties, due to the lower development of current wastewater treatment facilities, but that these areas consequently have the most potential for risk mitigation. This pattern intensifies under higher population growth scenarios. With this prototype, we developed a hierarchical probabilistic model and demonstrated its potential in forecasting the environmental risk of chemical stressors under plausible demographic and management scenarios, contributing to the further development of BNs for ERA. Integr Environ Assess Manag 2024;00:1-21. © 2024 The Authors. Integrated Environmental Assessment and Management published by Wiley Periodicals LLC on behalf of Society of Environmental Toxicology & Chemistry (SETAC).

SC-Track: a robust cell-tracking algorithm for generating accurate single-cell lineages from diverse cell segmentations.

Computational analysis of fluorescent timelapse microscopy images at the single-cell level is a powerful approach to study cellular changes that dictate important cell fate decisions. Core to this approach is the need to generate reliable cell segmentations and classifications necessary for accurate quantitative analysis. Deep learning-based convolutional neural networks (CNNs) have emerged as a promising solution to these challenges. However, current CNNs are prone to produce noisy cell segmentations and classifications, which is a significant barrier to constructing accurate single-cell lineages. To address this, we developed a novel algorithm called Single Cell Track (SC-Track), which employs a hierarchical probabilistic cache cascade model based on biological observations of cell division and movement dynamics. Our results show that SC-Track performs better than a panel of publicly available cell trackers on a diverse set of cell segmentation types. This cell-tracking performance was achieved without any parameter adjustments, making SC-Track an excellent generalized algorithm that can maintain robust cell-tracking performance in varying cell segmentation qualities, cell morphological appearances and imaging conditions. Furthermore, SC-Track is equipped with a cell class correction function to improve the accuracy of cell classifications in multiclass cell segmentation time series. These features together make SC-Track a robust cell-tracking algorithm that works well with noisy cell instance segmentation and classification predictions from CNNs to generate accurate single-cell lineages and classifications.

A hierarchical probabilistic underwater image enhancement model with reinforcement tuning

Underwater Image Enhancement (UIE) is a challenging problem due to the complex underwater environment. Traditional UIE methods can hardly adapt to various underwater environments. Deep learning-based UIE methods are more powerful but often rely on a large deal of real-world underwater images with distortion-free reference images. This gives rise to two issues: First, the reference images are highly uncertain because the ground-truth images cannot be are captured directly in underwater environment. Second, learning-based methods may lack generalization ability for diverse underwater environments. To tackle these issues, we propose HPUIE-RL, a hierarchical probabilistic UIE model facilitated by reinforcement learning. This model integrates UNet with hierarchical probabilistic modules to produce various enhanced candidate images that reflect the uncertainty of the enhancement. Then, a reinforcement learning fine-tuning framework is designed to fine-tune the pretrained model in an unsupervised manner, which responds to the dynamic underwater environment. Experiments on real-world datasets from diverse underwater environments demonstrate that our HPUIE-RL model outperforms state-of-the-art UIE methods regarding visual and quantitative performance and generalizability.

Hierarchical Attention-Based Age Estimation and Bias Analysis.

In this work, we present a Deep Learning approach to estimate age from facial images. First, we introduce a novel attention-based approach to image augmentation-aggregation, which allows multiple image augmentations to be adaptively aggregated using a Transformer-Encoder. A hierarchical probabilistic regression model is then proposed that combines discrete probabilistic age estimates with an ensemble of regressors. Each regressor is adapted and trained to refine the probability estimate over a given age range. We show that our age estimation scheme outperforms current schemes and provides a new state-of-the-art age estimation accuracy when applied to the MORPH II and CACD datasets. We also present an analysis of the biases in the results of the state-of-the-art age estimates.

Unsupervised phoneme and word acquisition from continuous speech based on a hierarchical probabilistic generative model

Humans can divide the perceived continuous speech signals, which exhibit double articulation structure, into phonemes and words without explicit boundary points or labels and thus learn a language. In constructive developmental studies, learning the double articulation structure of speech signals is important for realizing robots with human-like language learning abilities. In this study, we propose a novel probabilistic generative model called the Gaussian process-hidden semi-Markov model-based double articulation analyzer (GP-HSMM-DAA), which can learn phonemes and words from continuous speech signals by hierarchically connecting two probabilistic generative models (PGMs), namely, the Gaussian process-hidden semi-Markov model and hidden semi-Markov model. In the proposed model, the parameters of each PGM are mutually and complementarily updated and learned, enabling accurate learning of the phonemes and words. The experimental results reveal that GP-HSMM-DAA can segment continuous speech into phonemes and words with higher accuracy than the conventional method.

Stochastic modeling and statistical calibration with model error and scarce data

This paper introduces a procedure to assess the predictive accuracy of stochastic models subject to model error and sparse data. Model error is introduced as uncertainty on the coefficients of appropriate polynomial chaos expansions (PCE). The error associated with finite sample size allows us to conceive of these coefficients as statistics of the data that we describe as random variables whose influence on output quantities of interest is evaluated through the extended polynomial chaos expansion (EPCE). A Bayesian data assimilation scheme is introduced to update these expansions by considering the resulting nested chaos expansion as a hierarchical probabilistic model. Stochastic models of quantities of interest (QoI) are thus constructed and efficiently evaluated. The Metropolis–Hastings Markov chain Monte Carlo procedure is used to sample the posterior. Two illustrative analytical and numerical problems are used to demonstrate the proposed approach.

Learning Hierarchical Document Graphs From Multilevel Sentence Relations.

Organizing the implicit topology of a document as a graph, and further performing feature extraction via the graph convolutional network (GCN), has proven effective in document analysis. However, existing document graphs are often restricted to expressing single-level relations, which are predefined and independent of downstream learning. A set of learnable hierarchical graphs are built to explore multilevel sentence relations, assisted by a hierarchical probabilistic topic model. Based on these graphs, multiple parallel GCNs are used to extract multilevel semantic features, which are aggregated by an attention mechanism for different document-comprehension tasks. Equipped with variational inference, the graph construction and GCN are learned jointly, allowing the graphs to evolve dynamically to better match the downstream task. The effectiveness and efficiency of the proposed multilevel sentence relation graph convolutional network (MuserGCN) is demonstrated via experiments on document classification, abstractive summarization, and matching.

Triqler for Protein Summarization of Data from Data-Independent Acquisition Mass Spectrometry.

A frequent goal, or subgoal, when processing data from a quantitative shotgun proteomics experiment is a list of proteins that are differentially abundant under the examined experimental conditions. Unfortunately, obtaining such a list is a challenging process, as the mass spectrometer analyzes the proteolytic peptides of a protein rather than the proteins themselves. We have previously designed a Bayesian hierarchical probabilistic model, Triqler, for combining peptide identification and quantification errors into probabilities of proteins being differentially abundant. However, the model was developed for data from data-dependent acquisition. Here, we show that Triqler is also compatible with data-independent acquisition data after applying minor alterations for the missing value distribution. Furthermore, we find that it has better performance than a set of compared state-of-the-art protein summarization tools when evaluated on data-independent acquisition data.

Assessing Ecosystem State Space Models: Identifiability and Estimation

Hierarchical probability models are being used more often than non-hierarchical deterministic process models in environmental prediction and forecasting, and Bayesian approaches to fitting such models are becoming increasingly popular. In particular, models describing ecosystem dynamics with multiple states that are autoregressive at each step in time can be treated as statistical state space models (SSMs). In this paper, we examine this subset of ecosystem models, embed a process-based ecosystem model into an SSM, and give closed form Gibbs sampling updates for latent states and process precision parameters when process and observation errors are normally distributed. Here, we use simulated data from an example model (DALECev) and study the effects changing the temporal resolution of observations on the states (observation data gaps), the temporal resolution of the state process (model time step), and the level of aggregation of observations on fluxes (measurements of transfer rates on the state process). We show that parameter estimates become unreliable as temporal gaps between observed state data increase. To improve parameter estimates, we introduce a method of tuning the time resolution of the latent states while still using higher-frequency driver information and show that this helps to improve estimates. Further, we show that data cloning is a suitable method for assessing parameter identifiability in this class of models. Overall, our study helps inform the application of state space models to ecological forecasting applications where (1) data are not available for all states and transfers at the operational time step for the ecosystem model and (2) process uncertainty estimation is desired.

Learning global spatial information for multi-view object-centric models

Recently, several studies have been working on multi-view object-centric models, which predict unobserved views of a scene and infer object-centric representations from several observation views. In general, multi-object scenes can be uniquely determined if both the properties of individual objects and the spatial arrangement of objects are specified; however, existing multi-view object-centric models only infer object-level representations and lack spatial information. This insufficient modeling can degrade novel-view synthesis quality and make it difficult to generate novel scenes. We can model both spatial information and object representations by introducing a hierarchical probabilistic model, which contains a global latent variable on top of object-level latent variables. However, how to execute inference and training with that hierarchical multi-view object-centric model is unclear. Therefore, we introduce several crucial components which help inference and training with the proposed model. We show that the proposed method achieves good inference quality and can also generate novel scenes properly.

Missing Structural Health Monitoring Data Recovery Based on Bayesian Matrix Factorization

The exposure of bridge health-monitoring systems to extreme conditions often results in missing data, which constrains the health monitoring system from working. Therefore, there is an urgent need for an efficient data cleaning method. With the development of big data and machine-learning techniques, several methods for missing-data recovery have emerged. However, optimization-based methods may experience overfitting and demand extensive tuning of parameters, and trained models may still have substantial errors when applied to unseen datasets. Furthermore, many methods can only process monitoring data from a single sensor at a time, so the spatiotemporal dependence among monitoring data from different sensors cannot be extracted to recover missing data. Monitoring data from multiple sensors can be organized in the form of matrix. Therefore, matrix factorization is an appropriate way to handle monitoring data. To this end, a hierarchical probabilistic model for matrix factorization is formulated under a fully Bayesian framework by incorporating a sparsity-inducing prior over spatiotemporal factors. The spatiotemporal dependence is modeled to reconstruct the monitoring data matrix to achieve the missing-data recovery. Through experiments using continuous monitoring data of an in-service bridge, the proposed method shows good performance of missing-data recovery. Furthermore, the effect of missing data on the preset rank of matrix is also investigated. The results show that the model can achieve higher accuracy of missing-data recovery with higher preset rank under the same case of missing data.

DragNet: Learning-based deformable registration for realistic cardiac MR sequence generation from a single frame.

Deformable image registration (DIR) can be used to track cardiac motion. Conventional DIR algorithms aim to establish a dense and non-linear correspondence between independent pairs of images. They are, nevertheless, computationally intensive and do not consider temporal dependencies to regulate the estimated motion in a cardiac cycle. In this paper, leveraging deep learning methods, we formulate a novel hierarchical probabilistic model, termed DragNet, for fast and reliable spatio-temporal registration in cine cardiac magnetic resonance (CMR) images and for generating synthetic heart motion sequences. DragNet is a variational inference framework, which takes an image from the sequence in combination with the hidden states of a recurrent neural network (RNN) as inputs to an inference network per time step. As part of this framework, we condition the prior probability of the latent variables on the hidden states of the RNN utilised to capture temporal dependencies. We further condition the posterior of the motion field on a latent variable from hierarchy and features from the moving image. Subsequently, the RNN updates the hidden state variables based on the feature maps of the fixed image and the latent variables. Different from traditional methods, DragNet performs registration on unseen sequences in a forward pass, which significantly expedites the registration process. Besides, DragNet enables generating a large number of realistic synthetic image sequences given only one frame, where the corresponding deformations are also retrieved. The probabilistic framework allows for computing spatio-temporal uncertainties in the estimated motion fields. Our results show that DragNet performance is comparable with state-of-the-art methods in terms of registration accuracy, with the advantage of offering analytical pixel-wise motion uncertainty estimation across a cardiac cycle and being a motion generator. We will make our code publicly available.

Retraction Note: Electronic commerce commodity entity identities based on hierarchical probability model

Retraction Note: Electronic commerce commodity entity identities based on hierarchical probability model

Learning Rat-Like Behavior for a Small-Scale Biomimetic Robot

Existing biomimetic robots can perform some basic rat-like movement primitives (MPs) and simple behavior with stiff combinations of these MPs. To mimic typical rat behavior with high similarity, we propose parameterizing the behavior using a probabilistic model and movement characteristics. First, an analysis of fifteen 10min video sequences revealed that an actual rat has six typical behaviors in the open field, and each kind of behavior contains different bio-inspired combinations of eight MPs. We used the softmax classifier to obtain the behavior-movement hierarchical probability model. Secondly, we specified the MPs using movement parameters that are static and dynamic. We obtained the predominant values of the static and dynamic movement parameters using hierarchical clustering and fuzzy C-means clustering, respectively. These predominant parameters were used for fitting the rat spinal joint trajectory using a second-order Fourier series, and the joint trajectory was generalized using a back propagation neural network with two hidden layers. Finally, the hierarchical probability model and the generalized joint trajectory were mapped to the robot as control policy and commands, respectively. We implemented the six typical behaviors on the robot, and the results show high similarity when compared with the behaviors of actual rats.

Constraining the SN Ia host galaxy dust law distribution and mass step: hierarchical BayeSN analysis of optical and near-infrared light curves

ABSTRACT We use the BayeSN hierarchical probabilistic SED model to analyse the optical–NIR (BVriYJH) light curves of 86 Type Ia supernovae (SNe Ia) from the Carnegie Supernova Project to investigate the SN Ia host galaxy dust law distribution and correlations between SN Ia Hubble residuals and host mass. Our Bayesian analysis simultaneously constrains the mass step and dust RV population distribution by leveraging optical–NIR colour information. We demonstrate how a simplistic analysis where individual RV values are first estimated for each SN separately, and then the sample variance of these point estimates is computed, overestimates the RV population variance $\sigma _R^2$. This bias is exacerbated when neglecting residual intrinsic colour variation beyond that due to light curve shape. Instead, Bayesian shrinkage estimates of σR are more accurate, with fully hierarchical analysis of the light curves being ideal. For the 75 SNe with low-to-moderate reddening (peak apparent B − V ≤ 0.3), we estimate an RV distribution with population mean μR = 2.59 ± 0.14, and standard deviation σR = 0.62 ± 0.16. Splitting this subsample at the median host galaxy mass (1010.57 M⊙) yields consistent estimated RV distributions between low- and high-mass galaxies, with μR = 2.79 ± 0.18, σR = 0.42 ± 0.24, and μR = 2.35 ± 0.27, σR = 0.74 ± 0.36, respectively. When estimating distances from the full optical–NIR light curves while marginalizing over various forms of the dust RV distribution, a mass step of ≳0.06 mag persists in the Hubble residuals at the median host mass.

Helmholtz machine with differential privacy

Helmholtz machine(HM) is the classic hierarchical probabilistic model for building the probability distribution of perception data, and the wake-sleep(WS) algorithm has been widely used as a training algorithm. To prevent the attacker from restoring the training set data by using the trained HM model, we introduce a Gaussian mechanism to WS algorithm to propose wake-sleep algorithm based on differential privacy (DP-WS) and use DP-WS to train HM to get HM model with privacy protection, named DP-HM. We provide rigorous proof of the privacy guarantee. In addition, our experiments on MNIST and BioID face datasets show that DP-HM model can be trained under a modest privacy budget and still have acceptable model quality.

Profile Decomposition Based Hybrid Transfer Learning for Cold-Start Data Anomaly Detection

Anomaly detection is an essential task for quality management in smart manufacturing. An accurate data-driven detection method usually needs enough data and labels. However, in practice, there commonly exist newly set-up processes in manufacturing, and they only have quite limited data available for analysis. Borrowing the name from the recommender system, we call this process a cold-start process. The sparsity of anomaly, the deviation of the profile, and noise aggravate the detection difficulty. Transfer learning could help to detect anomalies for cold-start processes by transferring the knowledge from more experienced processes to the new processes. However, the existing transfer learning and multi-task learning frameworks are established on task- or domain-level relatedness. We observe instead, within a domain, some components (background and anomaly) share more commonality, others (profile deviation and noise) not. To this end, we propose a more delicate component-level transfer learning scheme, i.e., decomposition-based hybrid transfer learning ( DHTL ): It first decomposes a domain (e.g., a data source containing profiles) into different components (smooth background, profile deviation, anomaly, and noise); then, each component’s transferability is analyzed by expert knowledge; Lastly, different transfer learning techniques could be tailored accordingly. We adopted the Bayesian probabilistic hierarchical model to formulate parameter transfer for the background, and “ L 2,1 + L 1 ”-norm to formulate low dimension feature-representation transfer for the anomaly. An efficient algorithm based on Block Coordinate Descend is proposed to learn the parameters. A case study based on glass coating pressure profiles demonstrates the improved accuracy and completeness of detected anomaly, and a simulation demonstrates the fidelity of the decomposition results.

A fast collapsed Gibbs sampler for frequency domain operational modal analysis

This paper introduces a fast Gibbs sampler for solving a fully Bayesian problem in operational modal analysis. The proposed method is able to infer modal properties from the FFT of well-separated modes. System identification and related uncertainties are captured by a posterior distribution. The classical resonance description is wrapped into a hierarchical probabilistic model. The model is sampled through an enhanced Gibbs sampler including a Metropolis–Hasting step. The entire sampling scheme is new and the fast convergence of the algorithm is enabled by two strategies. First, the use of a collapsed Gibbs algorithm allows for an efficient sampling of the mode shape. Second, the use of adequate candidate distributions in the Metropolis–Hasting step provides excellent acceptance ratios, around 75%. Eventually, the numerical inference procedure is compared to the state-of-the-art Fast Bayesian FFT Algorithm (Fast-BFFTA), which is the most commonly used algorithm for Bayesian operational modal analysis. The sampler surpasses the Fast-BFFTA for small data-based identification, while remaining sustainable in terms of computing requirements.

Few-Shot Anomaly Detection Using Deep Generative Models for Grouped Data

There exists a great demand for automatic anomaly detection in industrial world. The anomaly has been defined as a group of samples that rarely or never appears. Given a type of products, one has to collect numerous samples and train an anomaly detector. When one diverts a model trained with old types of products with sufficient inventory to the new type, one can detect anomalies of the new type before a production line is established. However, because of the definition of the anomaly, a typical anomaly detector considers the new type of products anomalous even if it is consistent with the standard. Given the above practical demand, this study propose a novel problem setting, few-shot anomaly detection, where an anomaly detector trained in source domains is adapted to a small set of target samples without full retraining. Then, we tackle this problem using a hierarchical probabilistic model based on deep learning. Our empirical results on toy and real-world datasets demonstrate that the proposed model detects anomalies in a small set of target samples successfully.

Inconsistency-Tolerant Hierarchical Probabilistic CTL Model Checking: Logical Foundations and Illustrative Examples

In this study, an inconsistency-tolerant hierarchical probabilistic computation tree logic (IHpCTL) is developed to establish a new extended model-checking paradigm referred to as IHpCTL model checking, which is intended to verify randomized, open, large, and complex concurrent systems. The proposed IHpCTL is constructed based on several previously established extensions of the standard probabilistic temporal logic known as probabilistic computation tree logic (pCTL), which is widely used for probabilistic model checking. IHpCTL is shown to be embeddable into pCTL and is relatively decidable with respect to pCTL. This means that the decidability of pCTL with certain probability measures implies the decidability of IHpCTL. The results indicate that we can effectively reuse the previously proposed pCTL model-checking algorithms for IHpCTL model checking. Moreover, in this study, some new illustrative examples for clinical reasoning verification are addressed based on IHpCTL model checking.