Bayesian Model Fusion Research Articles

Investigating explainable transfer learning for battery lifetime prediction under state transitions

Battery lifetime prediction at early cycles is crucial for researchers and manufacturers to examine product quality and promote technology development. Machine learning has been widely utilized to construct data-driven solutions for high-accuracy predictions. However, the internal mechanisms of batteries are sensitive to many factors, such as charging/discharging protocols, manufacturing/storage conditions, and usage patterns. These factors will induce state transitions, thereby decreasing the prediction accuracy of data-driven approaches. Transfer learning is a promising technique that overcomes this difficulty and achieves accurate predictions by jointly utilizing information from various sources. Hence, we develop two transfer learning methods, Bayesian Model Fusion and Weighted Orthogonal Matching Pursuit, to strategically combine prior knowledge with limited information from the target dataset to achieve superior prediction performance. From our results, our transfer learning methods reduce root-mean-squared error by 41% through adapting to the target domain. Furthermore, the transfer learning strategies identify the variations of impactful features across different sets of batteries and therefore disentangle the battery degradation mechanisms and the root cause of state transitions from the perspective of data mining. These findings suggest that the transfer learning strategies proposed in our work are capable of acquiring knowledge across multiple data sources for solving specialized issues.

Correlated Bayesian Model Fusion: Efficient High-Dimensional Performance Modeling of Analog/RF Integrated Circuits Over Multiple Corners

Efficient high-dimensional performance modeling of analog/RF circuits over multiple corners is an important-yet-challenging task. In this article, we propose a novel performance modeling approach for analog/RF circuits, referred to as correlated Bayesian model fusion (C-BMF). The key idea is to encode the correlation information for both model template and coefficient magnitude among different corners by using a unified prior distribution. Next, the prior distribution is combined with a few simulation samples via Bayesian inference to efficiently determine the unknown model coefficients. Two circuit examples designed in a commercial 40-nm CMOS process demonstrate that C-BMF achieves about <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2\times $ </tex-math></inline-formula> cost reduction over the traditional state-of-the-art modeling technique without surrendering any accuracy.

Developing a Long Short-Term Memory-based signal processing method for Coriolis mass flowmeter

Coriolis mass flowmeter is widely used in various fields due to its high accuracy, but it still needs to be improved in some special conditions. This paper proposes a deep learning-based signal processing method for Coriolis mass flowmeter. Firstly, we set up an experimental platform to collect data, taking the vibration signal as the input feature and the mass flow as the sample label. Secondly, we designed networks with different structures (including LSTM, RNN and ANN) and adopted batch normalization to speed up convergence. Finally, Bayesian model fusion and moving average were used to reduce generalization error. Experiment results prove that the model with LSTM layer is better than other single models and the mean square error of the optimized model reduces to 0.0047, which is far superior to the calibrated meter (0.1200). These findings that get rid of traditional methods are expected to break through existing bottlenecks.

Distributed parallel deep learning of Hierarchical Extreme Learning Machine for multimode quality prediction with big process data

In this work, the distributed and parallel Extreme Learning Machine (dp-ELM) and Hierarchical Extreme Learning Machine (dp-HELM) are proposed for multimode process quality prediction with big data. The efficient ELM algorithm is transformed into the distributed and parallel modeling form according to the MapReduce framework. Since the deep learning network structure of HELM is more accurate than the single layer of ELM in feature representation, the dp-HELM is further developed through decomposing the ELM-based Auto-encoders (ELM-AE) of deep hidden layers into a loop of MapReduce jobs. Additionally, the multimode issue is solved through the “divide and rule” strategy. The distributed and parallel K-means (dp-K-means) is utilized to divide the process modes, which are further trained in a synchronous parallel way by dp-ELM and dp-HELM. Finally, the Bayesian model fusion technique is utilized to integrate the local models for online prediction. The proposed algorithms are deployed on a Hadoop MapReduce computing cluster and the feasibility and efficiency are illustrated through building a real industrial quality prediction model with big process data.

Bayesian Model Fusion: Large-Scale Performance Modeling of Analog and Mixed-Signal Circuits by Reusing Early-Stage Data

Efficient performance modeling of today’s analog and mixed-signal circuits is an important yet challenging task, due to the high-dimensional variation space and expensive circuit simulation. In this paper, we propose a novel performance modeling algorithm that is referred to as Bayesian model fusion (BMF) to address this challenge. The key idea of BMF is to borrow the information collected from an early stage (e.g., schematic level) to facilitate efficient performance modeling at a late stage (e.g., post layout). Such a goal is achieved by statistically modeling the performance correlation between early and late stages through Bayesian inference. Furthermore, to make the proposed BMF method of practical utility, four implementation issues, including: 1) prior mapping; 2) missing prior knowledge; 3) fast solver; and 4) prior and hyper-parameter selection, are carefully considered in this paper. Two circuit examples designed in a commercial 32 nm CMOS silicon on insulator process demonstrate that the proposed BMF method achieves up to $9\times $ runtime speed-up over the traditional modeling technique without surrendering any accuracy.

Bayesian Model Fusion for Forecasting Civil Unrest

With the rapid rise in social media, alternative news sources, and blogs, ordinary citizens have become information producers as much as information consumers. Highly charged prose, images, and videos spread virally, and stoke the embers of social unrest by alerting fellow citizens to relevant happenings and spurring them into action. We are interested in using Big Data approaches to generate forecasts of civil unrest from open source indicators. The heterogenous nature of data coupled with the rich and diverse origins of civil unrest call for a multi-model approach to such forecasting. We present a modular approach wherein a collection of models use overlapping sources of data to independently forecast protests. Fusion of alerts into one single alert stream becomes a key system informatics problem and we present a statistical framework to accomplish such fusion. Given an alert from one of the numerous models, the decision space for fusion has two possibilities: (i) release the alert or (ii) suppress the alert. Using a Bayesian decision theoretical framework, we present a fusion approach for releasing or suppressing alerts. The resulting system enables real-time decisions and more importantly tuning of precision and recall. Supplementary materials for this article are available online.

Multi-model fusion and error parameter estimation

A robust and practical methodology for multi-model ocean forecast fusion has been sought. Present regional ocean forecasting systems adapt and evolve in response to modelled processes. This makes it imperative that a forecast combination methodology be adaptive and capable to operate with a small sample of past validating events. To this end, we consider an extension of maximum-likelihood error parameter estimation to multi-model predictive systems, and utilize the resulting methodology for adaptive Bayesian model fusion. The methodology consists of the following three general steps: (a) parametrization of forecast uncertainties through either a suitable parametric family (e.g. covariance models) or through a low-rank approximation (e.g. flow-dependent error subspaces); (b) update of uncertainty parameters via maximum likelihood; and (c) combining model forecasts based on their uncertainty parameters via maximum likelihood. In order to implement step (b), we have extended the maximum-likelihood error parameter estimation methodology to multi-model forecasting systems using the expectation-maximization technique, with the true state-space vector at observation locations treated as missing data. With only one forecasting model, the procedure reduces to the standard maximum-likelihood error parameter estimation. The proposed multi-model fusion methodology neglects cross-model error correlations in order to gain the capability to work with a small sample of past events. We illustrate the methodology with a two-model forecasting example (HOPS, ROMS) within the framework of the real-time forecasting experiment held in Monterey Bay during 2003. Copyright © 2005 Royal Meteorological Society