Variational Bayes Expectation-maximization Algorithm Research Articles

Improvement of virtual in-situ calibration in air handling unit using data preprocessing based on Gaussian mixture model

Fault diagnosis of sensors used in air conditioning systems can significantly reduce the energy consumption and improve the indoor comfort. Previously, a virtual in-situ calibration method based on the Bayesian inference and Monte Carlo method was proposed to diagnose sensor faults without installing redundant sensors. However, air conditioning systems operate in continuously changing operating conditions because of the changes in the outdoor environment and indoor load. This dynamic nature of the operating conditions poses difficulty in the selection of the existing historical datasets and results in uncertainty in calibration results. To address this problem, this study proposes Gaussian mixture model (GMM) to preprocess historical data to obtain steady-state measurements under various operating conditions. A variational Bayes expectation maximization (VBEM) algorithm was applied to the GMM to increase its clustering accuracy and remove human intervention. Next, the expectation maximization (EM) algorithm was used to solve the GMM in order to illustrate that unsuccessful clustering will lead to fluctuation of calibration results and verify the accuracy of VBEM. In this case, the number of GMMs was determined manually based on attempt step by step. Finally, the clustering results of the two algorithms were incorporated into a virtual in-situ calibration method. Based on comparison of calibration results, it can be concluded that the calibration results deviated significantly from the true value when the number of GMMs was unreasonable after the EM algorithm. However, the accuracy of all sensors was improved by 75% after calibration, and convergence occurred when the datasets were two to four times the number of sensors after the VBEM algorithm.

Nonlinear industrial soft sensor development based on semi-supervised probabilistic mixture of extreme learning machines

Soft sensors play an important role in process industries for monitoring and control of key quality variables, and calibration of analyzers. Owing to the merits of fast learning speed and good generalization performance, extreme learning machines (ELMs) have been widely accepted to develop soft sensor models for nonlinear industrial processes. However, there still exist some challenges in developing high-accuracy ELM-based soft sensors. Specifically, ELMs with shallow networks seem to have inadequate representation capabilities for complex nonlinearities, while ELMs with deep networks have difficulties in determining the number of hidden layers and hidden nodes for each layer which readily results in overfitting. In addition, in soft sensor applications, labeled samples are usually limited due to technical or economical reasons, which adds obstacles to model training. To deal with these issues, we propose a semi-supervised probabilistic mixture of ELMs (referred to as the ‘S2PMELMs’). In the S2PMELMs, localized ELMs are trained and combined, which are completed in a unified probabilistic way such that process nonlinearities and uncertainties can be accommodated. Moreover, based on the variational Bayes expectation–maximization algorithm, we develop a training algorithm for the S2PMELMs, where unlabeled samples are able to be exploited and the regularization parameter for each ELM can be adaptively determined. The performance of the S2PMELMs is evaluated through two real-world industrial processes, and the results demonstrate the advantages of the proposed method in contrast with several state-of-the-art relevant soft sensing approaches.

Clustered fractional Gabor transform

In this paper, a novel clustered fractional Gabor transform (CFrGT) is proposed by exploiting the continuity structure of the fractional Gabor spectrum. The fractional Gabor expansion is reformulated under a Bayesian framework with correlated priors. To encourage the nonzero or zero coefficients to cluster in a spatial consistent constraint, the Markov random field (MRF) model is incorporated as the prior for the support of the fractional Gabor spectrum. And then the variational Bayes expectation-maximization algorithm is used to approximate the posterior of the hidden variables and estimate the parameters of MRF model. The proposed algorithm achieves high time-frequency resolution and localization under noisy and low sampling scenarios. Both the synthetic and the real data experimental results verify the effectiveness and its superiority over the conventional fractional Gabor transform.

MIMO-OFDM系统中基于变分Bayes EM算法的联合符号检测与鲁棒Kalman信道跟踪

A new joint symbol detection and channel estimation algorithm is proposed for MIMO-OFDM systems over fast fading channels based on the Variational Bayes Expectation-Maximization (VBEM) algorithm and turbo principle. By applying VBEM algorithm,a modified List Sphere Decoder (LSD) and soft-input Kalman Filter (KF) are derived for symbol detection and channel estimation, respectively. The covariance matrices of channel estimation are considered in the symbol detector, and the covariances of symbols are taken into account in channel estimator. In fast time-varying channels, soft symbols with low reliability may introduce outliers in soft-input kalman filter. Since the KF lacks robustness to outliers, filter divergence may occur and gives rise to propagation and error floor phenomenon. In order to mitigate the outliers effect, a robust soft-input kalman filter under the VBEM framework is proposed based on the theory of robust statistics, which can keep robust channel-tracking ability in fast fading channels. Simulation results show that the proposed VBEM algorithm has better performance over the conventional VBEM algorithm and EM algorithm in fast time-varying multipath channels.

The random subgraph model for the analysis of an ecclesiastical network in Merovingian Gaul

In the last two decades many random graph models have been proposed to extract knowledge from networks. Most of them look for communities or, more generally, clusters of vertices with homogeneous connection profiles. While the first models focused on networks with binary edges only, extensions now allow to deal with valued networks. Recently, new models were also introduced in order to characterize connection patterns in networks through mixed memberships. This work was motivated by the need of analyzing a historical network where a partition of the vertices is given and where edges are typed. A known partition is seen as a decomposition of a network into subgraphs that we propose to model using a stochastic model with unknown latent clusters. Each subgraph has its own mixing vector and sees its vertices associated to the clusters. The vertices then connect with a probability depending on the subgraphs only, while the types of edges are assumed to be sampled from the latent clusters. A variational Bayes expectation-maximization algorithm is proposed for inference as well as a model selection criterion for the estimation of the cluster number. Experiments are carried out on simulated data to assess the approach. The proposed methodology is then applied to an ecclesiastical network in Merovingian Gaul. An R code, called Rambo, implementing the inference algorithm is available from the authors upon request.

Variational Bayesian inference and complexity control for stochastic block models

It is now widely accepted that knowledge can be acquired from networks by clustering their vertices according to the connection profiles. Many methods have been proposed and in this paper we concentrate on the Stochastic Block Model (SBM). The clustering of vertices and the estimation of SBM model parameters have been subject to previous work, and numerous inference strategies such as variational expectation maximization (EM) and classification EM have been proposed. However, SBM still suffers from a lack of criteria to estimate the number of components in the mixture. To our knowledge, only one model-based criterion, Integrated Complete-data Likelihood (ICL), has been derived for SBM in the literature. It relies on an asymptotic approximation of the integrated complete-data likelihood and recent studies have shown that it tends to be too conservative in the case of small networks. To tackle this issue, we propose a new criterion that we call Integrated Likelihood Variational Bayes (ILvb), based on a non-asymptotic approximation of the marginal likelihood. We describe how the criterion can be computed through a variational Bayes EM algorithm.

Blind source separation with time series variational Bayes expectation maximization algorithm

This paper presents a variational Bayes expectation maximization algorithm for time series based on [email protected]? variational Bayesian theory. The proposed algorithm is applied in the blind source separation (BSS) problem to estimate both the source signals and the mixing matrix for the optimal model structure. The distribution of the mixing matrix is assumed to be a matrix Gaussian distribution due to the correlation of its elements and the inverse covariance of the sensor noise is assumed to be Wishart distributed for the correlation between sensor noises. The mixture of Gaussian model is used to approximate the distribution of each independent source. The rules to update the posterior hyperparameters and the posterior of the model structure are obtained. The optimal model structure is selected as the one with largest posterior. The source signals and mixing matrix are estimated by applying LMS and MAP estimators to the posterior distributions of the hidden variables and the model parameters respectively for the optimal structure. The proposed algorithm is tested with synthetic data. The results show that: (1) the logarithm posterior of the model structure increases with the accuracy of the posterior mixing matrix; (2) the accuracies of the prior mixing matrix, the estimated mixing matrix, and the estimated source signals increase with the logarithm posterior of the model structure. This algorithm is applied to Magnetoencephalograph data to localize the source of the equivalent current dipoles.