Continuous Time Bayesian Networks Research Articles

Structure learning for continuous time Bayesian networks via penalized likelihood

AbstractContinuous time Bayesian networks (CTBNs) represent a class of stochastic processes, which can be used to model complex phenomena, for instance, they can describe interactions occurring in living processes, social science models or medicine. The literature on this topic is usually focused on a case when a dependence structure of a system is known and we are to determine conditional transition intensities (parameters of a network). In the paper, we study a structure learning problem, which is a more challenging task and the existing research on this topic is limited. The approach, which we propose, is based on a penalized likelihood method. We prove that our algorithm, under mild regularity conditions, recognizes a dependence structure of a graph with high probability. We also investigate properties of the procedure in numerical studies.

Bayesian network approach for dynamic fault tree with common cause failures and interval uncertainty parameters

Traditional fault tree analysis often assumes that the basic events are independent and the failure parameters are known. Therefore, it is powerless to deal with the correlation among basic events and the uncertainty of failure parameters due to the small failure data. Therefore, a framework based on continuous-time Bayesian network is proposed to evaluate the reliability of fault tree with common cause failures (CCF) and uncertainty parameters. Firstly, the best-worst method (BWM) and hesitant fuzzy set (HFS) are introduced to address the issue of β-factor being influenced by experts’ subjectivity. Then, the interval theory is introduced to deal with the uncertainty parameters. Based on continuous-time Bayesian network, the conditional probability functions of logic gates (i.e. AND gate, OR gate, spare gate, priority AND gate) with CCF are derived, and the upper and lower bounds of failure probability of top event can be solved. Finally, the fault tree of CPU system is given to verify the effectiveness of the proposed framework.

Resilience evaluation of train control on-board system based on multi-dimensional continuous-time Bayesian network

The Bayesian network belongs to a commonly used model in the resilience research of Train Control on Board System (TCOBS). In the previous works, component failure is only considered to be affected by time, without showing the trend of resilience changing with time, and less attention paid to whether it can effectively recover to a usable state after failure. In this paper, we propose a multi-dimensional continuous-time Bayesian network to evaluate TCOBS resilience. Firstly, based on multi-dimensional influencing factors, the quantitative function of TCOBS component resilience is constructed. Secondly, the unit step function and impulse function are used to construct a multi-dimensional continuous-time Bayesian network, and the TCOBS resilience is derived and evaluated. A real-world case study is conducted by using the Chinese Train Control System-3 On-Board System as the background, two comparative experiments are conducted. The results show that, in addition to time factor, TCOBS resilience is also influenced by human and environmental factors. Therefore, to enhance the resilience of TCOBS, we should reduce failure rate and improve maintenance rate, and reduce the probability of human failure and environmental interference failure. The new model can find out the continuous trend of TCOBS resilience under the influence of multi-dimensional factors.

Resilience evaluation of train control on-board system considering common cause failure: Based on a beta-factor and continuous-time bayesian network model

Common cause failure is an inevitable factor in studying the reliability or resilience of a system with redundant structures. Train Control On Board System (TCOBS) is a typical redundant structural system. However, there are few studies on the resilience of TCOBS that consider the issue of common cause failure, which means the resilience assessment results might be imprecise. At the same time, previous TCOBS resilience evaluation works also have problems such as unreasonable resilience evaluation functions and low accuracy of resilience evaluation models. To address the above issues, in this paper, we firstly construct the Bayesian network structure of TCOBS and present the resilience function of TCOBS components. Secondly, a Beta-factor model is used to evaluate the common cause failure in TCOBS, and a continuous-time Bayesian network model is used to infer TCOBS resilience and the importance of each component. Finally, CTCS-3OBS is used as an example for case simulation and model comparison analysis. The results show that if the impact of common cause failure on TCOBS resilience is not considered, it will lead to a higher result of resilience assessment and significant differences in component importance assessment, both of which will lead to inaccurate formulation of TCOBS resilience improvement measures.

A Risk-Based Approach to Prognostics and Health Management Combining Bayesian Networks and Continuous-Time Bayesian Networks

A Risk-Based Approach to Prognostics and Health Management Combining Bayesian Networks and Continuous-Time Bayesian Networks

Constraint-based and hybrid structure learning of multidimensional continuous-time Bayesian network classifiers

Learning the structure of continuous-time Bayesian networks directly from data has traditionally been performed using score-based structure learning algorithms. Only recently has a constraint-based method been proposed, proving to be more suitable under specific settings, as in modelling systems with variables having more than two states. As a result, studying diverse structure learning algorithms is essential to learn the most appropriate models according to data characteristics and task-related priorities, such as learning speed or accuracy. This article proposes alternative algorithms for learning multidimensional continuous-time Bayesian network classifiers, introducing, for the first time, constraint-based and hybrid algorithms for these models. Nevertheless, these contributions also apply to the simpler one-dimensional classification problem for which only score-based solutions exist in the literature. More specifically, the aforementioned constraint-based structure learning algorithm is first adapted to the supervised classification setting. Then, a novel algorithm of this kind, specifically tailored for the multidimensional classification problem, is presented to improve the learning times for the induction of multidimensional classifiers. Finally, a hybrid algorithm is introduced, attempting to combine the strengths of the score- and constraint-based approaches. Experiments with synthetic and real-world data are performed not only to validate the capabilities of the proposed algorithms but also to conduct a comparative study of the available competitors.

Reliability and Service Life Analysis of Airbag Systems

Airbag systems are important to a car’s safety protection system. To further improve the reliability of the system, this paper analyzes the failure mechanism of automotive airbag systems and establishes a dynamic fault tree model. The dynamic fault tree model is transformed into a continuous-time Bayesian network by introducing a unit step function and an impulse function, from which the failure probability of the system is calculated. Finally, the system reliability and average life are calculated and analyzed and compared with the sequential binary decision diagram method. The results show that the method can obtain more accurate system reliability and effectively identify the weak parts of the automotive airbag system, to a certain extent compensating for the lack of computational complexity of dynamic Bayesian networks in solving system reliability problems with continuous failure processes.

A fatigue reliability assessment approach for wind turbine blades based on continuous time Bayesian network and FEA

AbstractWind turbine blades made by composite materials (CWTBs), encounter fatigue failures, such as cracks, fractures, delamination, etc. Finite Element Analysis (FEA) is applied for fatigue performance simulations of CWTBs as the full‐scale testing is costly. To consider correlated failures and uncertainties in load and material parameters, this paper proposes a fatigue reliability assessment method based on continuous time Bayesian network and FEA. Specifically, the dangerous regions of each component of CWTBs are determined by finite element fatigue simulation. The failure probability distributions of components are then computed by quantifying the uncertainties of several factors including the load and material parameters. A continuous time Bayesian network model is constructed for the fatigue reliability of CWTBs. The performance of the proposed method is verified by a comprehensive analysis with the results of discrete time Bayesian networks.

AN EVIDENCE-BASED RISK DECISION SUPPORT APPROACH FOR METRO TUNNEL CONSTRUCTION

The risk-informed decision-making of metro tunnel project is often faced with the problem of inadequate utilization of available information. In order to address the epistemic uncertainty problem caused by insufficient utilization of information in decision-making, this paper proposes a risk decision support approach for metro tunnel construction based on Continuous Time Bayesian Network (CTBN) technique. CTBN can factor the state space of variables in tunnel projects and perform evidence-based reasoning, which enables the diverse information of expert opinions, project-specific parameters, historical data and engineering anomalies to be the evidence to support decision-making. A concise CTBN model development method based on Dynamic Fault Trees is presented to replace the cumbersome model learning process. The proposed approach can utilize multi-source information as evidence to provide multi-form decision support both in the pre-construction stage and construction stage of the tunnel construction project, and the results can support the decisions on judging the acceptability of the risk, developing response strategies for risk factors and diagnosing the causes of the hazardous event. A case study on the water leakage risk of tunnel construction in China is presented to illustrate the feasibility of the approach. The case study shows that the approach can assist in making informed decisions, so as to improve the engineering safety.

Active learning of continuous-time Bayesian networks through interventions* *This article is an updated version of: Linzner D and Koeppl H 2021 Active learning of continuous-time Bayesian networks through interventions Proc. 38th Int. Conf. on Machine Learning vol 139 ed M Meila and T Zhang pp 6692–701.

We consider the problem of learning structures and parameters of continuous-time Bayesian networks (CTBNs) from time-course data under minimal experimental resources. In practice, the cost of generating experimental data poses a bottleneck, especially in the natural and social sciences. A popular approach to overcome this is Bayesian optimal experimental design (BOED). However, BOED becomes infeasible in high-dimensional settings, as it involves integration over all possible experimental outcomes. We propose a novel criterion for experimental design based on a variational approximation of the expected information gain. We show that for CTBNs, a semi-analytical expression for this criterion can be calculated for structure and parameter learning. By doing so, we can replace sampling over experimental outcomes by solving the CTBNs master-equation, for which scalable approximations exist. This alleviates the computational burden of integrating over possible experimental outcomes in high-dimensions. We employ this framework in order to recommend interventional sequences. In this context, we extend the CTBN model to conditional CTBNs in order to incorporate interventions. We demonstrate the performance of our criterion on synthetic and real-world data.

Multidimensional continuous time Bayesian network classifiers

The multidimensional classification of multivariate time series deals with the assignment of multiple classes to time-ordered data described by a set of feature variables. Although this challenging task has received almost no attention in the literature, it is present in a wide variety of domains, such as medicine, finance or industry. The complexity of this problem lies in two nontrivial tasks, the learning with multivariate time series in continuous time and the simultaneous classification of multiple class variables that may show dependencies between them. These can be addressed with different strategies, but most of them may involve a difficult preprocessing of the data, high space and classification complexity or ignoring useful interclass dependencies. Additionally, no attention has been given to the development of new multidimensional classifiers of time series based on probabilistic graphical models, even though transparent models can facilitate further understanding of the domain. In this paper, a novel probabilistic graphical model is proposed, which is able to classify a discrete multivariate temporal sequence into multiple class variables while modeling their dependencies. This model extends continuous time Bayesian networks to the multidimensional classification problem, which are able to explicitly represent the behavior of time series that evolve over continuous time. Different methods for the learning of the parameters and structure of the model are presented, and numerical experiments on synthetic and real-world data show encouraging results in terms of performance and learning time with respect to independent classifiers, the current alternative approach under the continuous time Bayesian network paradigm.

A constraint-based algorithm for the structural learning of continuous-time Bayesian networks

Dynamic Bayesian networks have been well explored in the literature as discrete-time models: however, their continuous-time extensions have seen comparatively little attention. In this paper, we propose the first implementation of a constraint-based algorithm for learning the structure of continuous-time Bayesian networks. We discuss the different statistical tests and the underlying hypotheses used by our proposal to establish conditional independence. Furthermore, we analyze and discuss the computational complexity of the best and worst cases for the proposed algorithm. Finally, we validate its performance using synthetic data, and we discuss its strengths and limitations comparing it with the score-based structure learning algorithm from Nodelman et al. [23]. We find the latter to be more accurate in learning networks with binary variables, while our constraint-based approach is more accurate with variables assuming more than two values. Numerical experiments confirm that score-based and constraint-based algorithms are comparable in terms of computation time.

A Functional Model for Structure Learning and Parameter Estimation in Continuous Time Bayesian Network: An Application in Identifying Patterns of Multiple Chronic Conditions.

Bayesian networks are powerful statistical models to study the probabilistic relationships among sets of random variables with significant applications in disease modeling and prediction. Here, we propose a continuous time Bayesian network with conditional dependencies represented as regularized Poisson regressions to model the impact of exogenous variables on the conditional intensities of the network. We also propose an adaptive group regularization method with an intuitive early stopping feature based on Gaussian mixture model clustering for efficient learning of the structure and parameters of the proposed network. Using a dataset of patients with multiple chronic conditions extracted from electronic health records of the Department of Veterans Affairs, we compare the performance of the proposed network with some of the existing methods in the literature for both short-term (one-year ahead) and long-term (multi-year ahead) predictions. The proposed model provides a sparse intuitive representation of the complex functional relationships between multiple chronic conditions. It also provides the capability of analyzing multiple disease trajectories over time, given any combination of preexisting conditions.

Dynamic Functional Continuous Time Bayesian Networks for Prediction and Monitoring of the Impact of Patients' Modifiable Lifestyle Behaviors on the Emergence of Multiple Chronic Conditions.

More than a quarter of all Americans are estimated to have multiple chronic conditions (MCC). It is known that shared modifiable lifestyle behaviors account for many common MCC. What is not precisely known is the dynamic effect of changes in lifestyle behaviors on the trajectories of MCC emergence. This paper proposes dynamic functional continuous time Bayesian networks to effectively formulate the dynamic effect of patients' modifiable lifestyle behaviors and their interaction with non-modifiable demographics and preexisting conditions on the emergence of MCC. The proposed method considers the parameters of the conditional dependencies of MCC as a nonlinear state-space model and develops an extended Kalman filter to capture the dynamics of the modifiable risk factors on the MCC evolution. It also develops a tensor-based control chart based on the integration of multilinear principal component analysis and multivariate exponentially weighted moving average chart to monitor the effect of changes in the modifiable risk factors on the risk of new MCC. We validate the proposed method based on a combination of simulation and a real dataset of 385 patients from the Cameron County Hispanic Cohort. The dataset examines the emergence of 5 chronic conditions (Diabetes, Obesity, Cognitive Impairment, Hyperlipidemia, Hypertension) based on 4 modifiable lifestyle behaviors representing (Diet, Exercise, Smoking Habits, Drinking Habits) and 3 non-modifiable demographic risk factors (Age, Gender, Education). For the simulated study, the proposed algorithm shows a run-length of 4 samples (4 months) to identify behavioral changes with significant impacts on the risk of new MCC. For the real data study, the proposed algorithm shows a run-length of one sample (one year) to identify behavioral changes with significant impacts on the risk of new MCC. The results demonstrate the sensitivity of the proposed methodology for dynamic prediction and monitoring of the risk of MCC emergence in individual patients.

Modeling Approaches Reveal New Regulatory Networks in Aspergillus fumigatus Metabolism.

Systems biology approaches are extensively used to model and reverse-engineer gene regulatory networks from experimental data. Indoleamine 2,3-dioxygenases (IDOs)—belonging in the heme dioxygenase family—degrade l-tryptophan to kynurenines. These enzymes are also responsible for the de novo synthesis of nicotinamide adenine dinucleotide (NAD+). As such, they are expressed by a variety of species, including fungi. Interestingly, Aspergillus may degrade l-tryptophan not only via IDO but also via alternative pathways. Deciphering the molecular interactions regulating tryptophan metabolism is particularly critical for novel drug target discovery designed to control pathogen determinants in invasive infections. Using continuous time Bayesian networks over a time-course gene expression dataset, we inferred the global regulatory network controlling l-tryptophan metabolism. The method unravels a possible novel approach to target fungal virulence factors during infection. Furthermore, this study represents the first application of continuous-time Bayesian networks as a gene network reconstruction method in Aspergillus metabolism. The experiment showed that the applied computational approach may improve the understanding of metabolic networks over traditional pathways.

Event-Driven Continuous Time Bayesian Networks

We introduce a novel event-driven continuous time Bayesian network (ECTBN) representation to model situations where a system's state variables could be influenced by occurrences of events of various types. In this way, the model parameters and graphical structure capture not only potential “causal” dynamics of system evolution but also the influence of event occurrences that may be interventions. We propose a greedy search procedure for structure learning based on the BIC score for a special class of ECTBNs, showing that it is asymptotically consistent and also effective for limited data. We demonstrate the power of the representation by applying it to model paths out of poverty for clients of CityLink Center, an integrated social service provider in Cincinnati, USA. Here the ECTBN formulation captures the effect of classes/counseling sessions on an individual's life outcome areas such as education, transportation, employment and financial education.

Listen to Your Face: Inferring Facial Action Units from Audio Channel

Extensive efforts have been devoted to recognizing facial action units (AUs). However, it is still challenging to recognize AUs from spontaneous facial displays especially when they are accompanied by speech. Different from all prior work that utilized visual observations for facial AU recognition, this paper presents a novel approach that recognizes speech-related AUs exclusively from audio signals based on the fact that facial activities are highly correlated with voice during speech. Specifically, dynamic and physiological relationships between AUs and phonemes are modeled through a continuous time Bayesian network (CTBN); then AU recognition is performed by probabilistic inference via the CTBN model. A pilot audiovisual AU-coded database has been constructed to evaluate the proposed audio-based AU recognition framework. The database consists of a “clean” subset with frontal and neutral faces and a challenging subset collected with large head movements and occlusions. Experimental results on this database show that the proposed CTBN model achieves promising recognition performance for 7 speech-related AUs and outperforms both the state-of-the-art visual-based and audio-based methods especially for those AUs that are activated at low intensities or “hardly visible” in the visual channel. The improvement is more impressive on the challenging subset, where the visual-based approaches suffer significantly.

Continuous Time Bayesian Networks in Prognosis and Health Management of Centrifugal Pumps

This paper presents a novel method for performing risk-based prognosis and health management (rPHM) on centrifugal pumps. We present the rPHM framework and apply common modeling tools used in reliability and testability analysis---dependency (D) matrices and fault tree analysis---as a basis for constructing an underlying predictive model. We then introduce the mathematics of the Continuous Time Bayesian Network (CTBN), which is a probabilistic graphical model based on a factored Markov process that is designed to capture system evolution through time, and we explain how to apply a CTBN derived from D-matrices and fault trees to consider the impact of a set of faults common to centrifugal pumps on emerging hazards in the pump system. We demonstrate the utility of using CTBNs for rPHM analysis with two experiments showing the descriptive power of our modeling approach.

Compact structures for continuous time Bayesian networks

The continuous time Bayesian network (CTBN) is a model capable of describing the probability distribution over a set of variables as it changes in time. The model relies on a directed graph structure to describe direct dependencies between variables, thereby simplifying the underlying parameters that describe the initial distribution and transition behavior. Although this approach can be effective in managing complexity, the size of the model can still be intractable if one variable depends directly on many other variables. To address this issue, we present two methods for imposing additional structure on the model that are capable of capturing regularities in the data that cannot be represented using the CTBN graph structure alone. We demonstrate how these methods can reduce model complexity and compare the representational capabilities of both approaches.

Reconstructing Activity Location Sequences From Incomplete Check-In Data: A Semi-Markov Continuous-Time Bayesian Network Model

Geo-location data from the check-ins made in online social media offers us information, in new ways, to understand activity-location choices of a large number of people. However, one of the major challenges of using check-in data is that it has missing activities, since users share their activities voluntarily. In this paper, we present a probabilistic modeling approach to reconstruct user activity-location sequences from this incomplete activity participation information. Specifically, we answer the question of how to predict an individual’s next activity, its duration and location given the incomplete trajectory data. The model describes the dynamics of individual activity participation behavior evolving over continuous time. A semi-Markov modeling approach is used to capture the stochastic processes involved in the activity generation mechanism. We present a particle-based Markov chain Monte Carlo sampler to run inference over the model. We further develop an expectation-maximization algorithm to learn the unknown parameters of the model from incomplete trajectory data. Finally, the method is applied to synthetically generated activity-location sequences and a data set of Foursquare check-ins of the users from New York City. Our experiments show that this method can successfully extract the true transition and duration distributions given the incomplete trajectory information. The proposed approach can help building many intelligent transportation applications using check-in data.