Dynamic Bayesian Network Learning Research Articles

Dynamic Bayesian network structure learning based on an improved bacterial foraging optimization algorithm

With the rapid development of artificial intelligence and data science, Dynamic Bayesian Network (DBN), as an effective probabilistic graphical model, has been widely used in many engineering fields. And swarm intelligence algorithm is an optimization algorithm based on natural selection with the characteristics of distributed, self-organization and robustness. By applying the high-performance swarm intelligence algorithm to DBN structure learning, we can fully utilize the algorithm's global search capability to effectively process time-based data, improve the efficiency of network generation and the accuracy of network structure. This study proposes an improved bacterial foraging optimization algorithm (IBFO-A) to solve the problems of random step size, limited group communication, and the inability to maintain a balance between global and local searching. The IBFO-A algorithm framework comprises four layers. First, population initialization is achieved using a logistics-sine chaotic mapping strategy as the basis for global optimization. Second, the activity strategy of a colony foraging trend is constructed by combining the exploration phase of the Osprey optimization algorithm. Subsequently, the strategy of bacterial colony propagation is improved using a "genetic" approach and the Multi-point crossover operator. Finally, the elimination-dispersal activity strategy is employed to escape the local optimal solution. To solve the problem of complex DBN learning structures due to the introduction of time information, a DBN structure learning method called IBFO-D, which is based on the IBFO-A algorithm framework, is proposed. IBFO-D determines the edge direction of the structure by combining the dynamic K2 scoring function, the designed V-structure orientation rule, and the trend activity strategy. Then, according to the improved reproductive activity strategy, the concept of "survival of the fittest" is applied to the network candidate solution while maintaining species diversity. Finally, the global optimal network structure with the highest score is obtained based on the elimination-dispersal activity strategy. Multiple tests and comparison experiments were conducted on 10 sets of benchmark test functions, two non-temporal and temporal data types, and six data samples of two benchmark 2T-BN networks to evaluate and analyze the optimization performance and structure learning ability of the proposed algorithm under various data types. The experimental results demonstrated that IBFO-A exhibits good convergence, stability, and accuracy, whereas IBFO-D is an effective approach for learning DBN structures from data and has practical value for engineering applications.

Pyramid Dynamic Bayesian Networks for Key Performance Indicator Prediction in Long Time-Delay Industrial Processes

Building Dynamic Bayesian Networks (DBN) for time-delay industrial processes has always been tough, since the structure learning of DBN is a NP-hard problem. In this article, a pyramid DBN framework is proposed to speed up modeling and improve feature learning for industrial processes with large time delays. In the pyramid DBN framework, a sequence of small-sized DBNs are established, each of which is a progressively simpler representation of the previous layer, and the feature information learned by the DBN sequence corresponds to the layers in the pyramid. With information fusion of the feature pyramid and a further feature filtering with hill climbing based on Bayesian information criterion, we can restore the feature information of the time-delay industrial processes. Regression models are built based on the features learned by the pyramid DBN framework for further Key Performance Indicator (KPI) estimation. Advantages of the method have been effectively validated on two actual industrial cases. With multi-level feature learning and filtering, both modeling speed and prediction accuracy have been greatly improved. The proposed method outperforms other similar works on identifying important features in industrial processes and existing DBN-based soft sensors.

Gat2Get: A Novel Approach to Infer Gene Regulatory Network from Gene Activity using Dynamic Bayesian Network learning

Gat2Get: A Novel Approach to Infer Gene Regulatory Network from Gene Activity using Dynamic Bayesian Network learning

Wideband Multitarget Tracking Based on Dynamic Bayesian Network Learning in an Acoustic Sensor Array Network

The multi-target tracking (MTT) based on distributed fusion methods in an acoustic sensor array network (ASAN) is limited by the performance of measurement parameter estimators, such as the received signal strength (RSS) and the direction-of-arrival (DOA). For measurement parameters with low accuracy and resolution, the MTT may fail in subsequent steps, e.g., data association, because the loss of upstream information cannot be made up by downstream processing. Thus, we propose a new wideband MTT algorithm based on dynamic Bayesian network (DBN), which treats the ASAN as an overall extended array, and directly estimates the target states from the raw acoustic data. The DBN fuses the near-field model, the acoustic propagation model and the motion model. These sub-models can be optimized by each other, improving the final estimation. Also, for each sub-band, target signals and the precision parameters of the sensor noise are treated as hidden random variables. Based on this, the weight of each sub-band can be automatically adjusted according to the accurate hidden variables. Besides, the optimization problem of the posterior probability is transformed into a graphical model learning problem. Moreover, for non-conjugate models, a novel algorithm based on Laplace approximations (LAs) with Newton’s method (NM) is developed, i.e., DBN-LA-NM, bypassing data association. In addition, the corresponding CRLB and convergence conditions are derived. The numerical simulation results show that the proposed algorithm outperforms existing MTTs based on the near-field model in terms of accuracy, convergence and computational complexity. Field experiments further verify the feasibility of the proposed algorithm.

Dynamic Bayesian Collective Awareness Models for a Network of Ego-Things

A novel approach is proposed for multimodal collective awareness (CA) of multiple networked intelligent agents. Each agent is here considered as an Internet-of-Things (IoT) node equipped with machine learning capabilities; CA aims to provide the network with updated causal knowledge of the state of execution of actions of each node performing a joint task, with particular attention to anomalies that can arise. Data-driven dynamic Bayesian models learned from multisensory data recorded during the normal realization of a joint task (agent network experience) are used for distributed state estimation of agents and detection of abnormalities. A set of switching dynamic Bayesian network (DBN) models collectively learned in a training phase, each related to particular sensorial modality, is used to allow each agent in the network to perform synchronous estimation of possible abnormalities occurring when a new task of the same type is jointly performed. Collective DBN (CDBN) learning is performed by unsupervised clustering of generalized errors (GEs) obtained from a starting generalized model. A growing neural gas (GNG) algorithm is used as a basis to learn the discrete switching variables at the semantic level. Conditional probabilities linking nodes in the CDBN models are estimated using obtained clusters. CDBN models are associated with a Bayesian inference method, namely, distributed Markov jump particle filter (D-MJPF), employed for joint state estimation and abnormality detection. The effects of networking protocols and of communications in the estimation of state and abnormalities are analyzed. Performance is evaluated by using a small network of two autonomous vehicles performing joint navigation tasks in a controlled environment. In the proposed method, first the sharing of observations is considered in ideal condition, and then the effects of a wireless communication channel have been analyzed for the collective abnormality estimation of the agents. Rician wireless channel and the usage of two protocols (i.e., IEEE 802.11p and IEEE 802.15.4) along with different channel conditions are considered as well.

The Hybrid parameter learning algorithm development for dynamic Bayesian network in the context of the Metropolis — Hastings approach

Bayesian networks are an efficient tool for analyzing complex stochastic processes. The generic methods of analysis based on Bayesian networks are probabilistic inference procedures that aimed at solving filtering, prediction and smoothing problems. An essential stage in the probabilistic inference methods implementation is network structure and parameters learning carried out on the special training samples obtained as a result of passive observation of the analyzed process and/or active experiment. Effectiveness of the Bayesian networks parameters learning algorithms directly depends on the preprocessing and organization of the training sample processing, which optimizes the classical learning algorithms capabilities. In the context of research introduced in this article, a hybrid algorithm for the parameters learning of dynamic Bayesian networks with observed and hidden variables has been developed. Algorithm allows increasing the efficiency of probabilities calculation procedure for the initial state and the probabilities of transition between the networks slices involving the variety of special methods for organizing training sample preprocessing. The questions of convergence of the proposed algorithm are also considered.

A New Approach to Learning in Dynamic Bayesian Networks (DBNs)

In this paper, we revisit the parameter learning problem, namely the estimation of model parameters for Dynamic Bayesian Networks (DBNs). DBNs are directed graphical models of stochastic processes that encompasses and generalize Hidden Markov models (HMMs) and Linear Dynamical Systems (LDSs). Whenever we apply these models to economics and finance, we are forced to make some modeling assumptions about the state dynamics and the graph topology (the DBN structure). These assumptions may be incorrectly specified and contain some additional noise compared to reality. Trying to use a best fit approach through maximum likelihood estimation may miss this point and try to fit at any price these models on data. We present here a new methodology that takes a radical point of view and instead focus on the final efficiency of our model. Parameters are hence estimated in terms of their efficiency rather than their distributional fit to the data. The resulting optimization problem that consists in finding the optimal parameters is a hard problem. We rely on Covariance Matrix Adaptation Evolution Strategy (CMA-ES) method to tackle this issue. We apply this method to the seminal problem of trend detection in financial markets. We see on numerical results that the resulting parameters seem less error prone to over fitting than traditional moving average cross over trend detection and perform better. The method developed here for algorithmic trading is general. It can be applied to other real case applications whenever there is no physical law underlying our DBNs.

Tractography-Based Score for Learning Effective Connectivity From Multimodal Imaging Data Using Dynamic Bayesian Networks.

Effective connectivity (EC) is the methodology for determining functional-integration among the functionally active segregated regions of the brain. By definition EC is "the causal influence exerted by one neuronal group on another" which is constrained by anatomical connectivity (AC) (axonal connections). AC is necessary for EC but does not fully determine it, because synaptic communication occurs dynamically in a context-dependent fashion. Although there is a vast emerging evidence of structure-function relationship using multimodal imaging studies, till date only a few studies have done joint modeling of the two modalities: functional MRI (fMRI) and diffusion tensor imaging (DTI). We aim to propose a unified probabilistic framework that combines information from both sources to learn EC using dynamic Bayesian networks (DBNs). DBNs are probabilistic graphical temporal models that learn EC in an exploratory fashion. Specifically, we propose a novel anatomically informed (AI) score that evaluates fitness of a given connectivity structure to both DTI and fMRI data simultaneously. The AI score is employed in structure learning of DBN given the data. Experiments with synthetic-data demonstrate the face validity of structure learning with our AI score over anatomically uninformed counterpart. Moreover, real-data results are cross-validated by performing classification-experiments. EC inferred on real fMRI-DTI datasets is found to be consistent with previous literature and show promising results in light of the AC present as compared to other classically used techniques such as Granger-causality. Multimodal analyses provide a more reliable basis for differentiating brain under abnormal/diseased conditions than the single modality analysis.

Recognizing multi-modal sensor signals using evolutionary learning of dynamic Bayesian networks

Multi-modal context-aware systems can provide user-adaptive services, but it requires complicated recognition models with larger resources. The limitations to build optimal models and infer the context efficiently make it difficult to develop practical context-aware systems. We developed a multi-modal context-aware system with various wearable sensors including accelerometers, gyroscopes, physiological sensors, and data gloves. The system used probabilistic models to handle the uncertain and noisy time-series sensor data. In order to construct the efficient probabilistic models, this paper uses an evolutionary algorithm to model structure and EM algorithm to determine parameters. The trained models are selectively inferred based on a semantic network which describes the semantic relations of the contexts and sensors. Experiments with the real data collected show the usefulness of the proposed method.

Efficient Structure Learning of Bayesian Networks using Constraints

This paper addresses the problem of learning Bayesian network structures from data based on score functions that are decomposable. It describes properties that strongly reduce the time and memory costs of many known methods without losing global optimality guarantees. These properties are derived for different score criteria such as Minimum Description Length (or Bayesian Information Criterion), Akaike Information Criterion and Bayesian Dirichlet Criterion. Then a branch-and-bound algorithm is presented that integrates structural constraints with data in a way to guarantee global optimality. As an example, structural constraints are used to map the problem of structure learning in Dynamic Bayesian networks into a corresponding augmented Bayesian network. Finally, we show empirically the benefits of using the properties with state-of-the-art methods and with the new algorithm, which is able to handle larger data sets than before.

Mocapy++ - A toolkit for inference and learning in dynamic Bayesian networks

BackgroundMocapy++ is a toolkit for parameter learning and inference in dynamic Bayesian networks (DBNs). It supports a wide range of DBN architectures and probability distributions, including distributions from directional statistics (the statistics of angles, directions and orientations).ResultsThe program package is freely available under the GNU General Public Licence (GPL) from SourceForge http://sourceforge.net/projects/mocapy. The package contains the source for building the Mocapy++ library, several usage examples and the user manual.ConclusionsMocapy++ is especially suitable for constructing probabilistic models of biomolecular structure, due to its support for directional statistics. In particular, it supports the Kent distribution on the sphere and the bivariate von Mises distribution on the torus. These distributions have proven useful to formulate probabilistic models of protein and RNA structure in atomic detail.

Adaptive learning of dynamic Bayesian networks with changing structures by detecting geometric structures of time series

A dynamic Bayesian network (DBN) is one of popular approaches for relational knowledge discovery such as modeling relations or dependencies, which change over time, between variables of a dynamic system. In this paper, we propose an adaptive learning method (autoDBN) to learn DBNs with changing structures from multivariate time series. In autoDBN, segmentation of time series is achieved first through detecting geometric structures transformed from time series, and then model regions are found from the segmentation by designed finding strategies; in each found model region, a DBN model is established by existing structure learning methods; finally, model revisiting is developed to refine model regions and improve DBN models. These techniques provide a special mechanism to find accurate model regions and discover a sequence of DBNs with changing structures, which are adaptive to changing relations between multivariate time series. Experimental results on simulated and real time series show that autoDBN is very effective in finding accurate/reasonable model regions and gives lower error rates, outperforming the switching linear dynamic system method and moving window method.

Adaptive learning of dynamic Bayesian networks with changing structures by detecting geometric structures of time series

Adaptive learning of dynamic Bayesian networks with changing structures by detecting geometric structures of time series

Boosted learning in dynamic bayesian networks for multimodal speaker detection

Bayesian network models provide an attractive framework for multimodal sensor fusion. They combine an intuitive graphical representation with efficient algorithms for inference and learning. However, the unsupervised nature of standard parameter learning algorithms for Bayesian networks can lead to poor performance in classification tasks. We have developed a supervised learning framework for Bayesian networks, which is based on the Adaboost algorithm of Schapire and Freund. Our framework covers static and dynamic Bayesian networks with both discrete and continuous states. We have tested our framework in the context of a novel multimodal HCI application: a speech-based command and control interface for a Smart Kiosk. We provide experimental evidence for the utility of our boosted learning approach.