PC Algorithm Research Articles

Partitioned hybrid learning of Bayesian network structures

We develop a novel hybrid method for Bayesian network structure learning called partitioned hybrid greedy search (pHGS), composed of three distinct yet compatible new algorithms: Partitioned PC (pPC) accelerates skeleton learning via a divide-and-conquer strategy, p-value adjacency thresholding (PATH) effectively accomplishes parameter tuning with a single execution, and hybrid greedy initialization (HGI) maximally utilizes constraint-based information to obtain a high-scoring and well-performing initial graph for greedy search. We establish structure learning consistency of our algorithms in the large-sample limit, and empirically validate our methods individually and collectively through extensive numerical comparisons. The combined merits of pPC and PATH achieve significant computational reductions compared to the PC algorithm without sacrificing the accuracy of estimated structures, and our generally applicable HGI strategy reliably improves the estimation structural accuracy of popular hybrid algorithms with negligible additional computational expense. Our empirical results demonstrate the competitive empirical performance of pHGS against many state-of-the-art structure learning algorithms.

A data-driven Bayesian network model integrating physical knowledge for prioritization of risk influencing factors

The coupling of multiple factors stemming from propagation effects and interdependency relationships among risks is prone to generate major accidents. It is of necessity to develop a feasible model with limited cases, which can generate reliable causal relationship evolution. To prioritize risk-influencing factors (RIFs) and investigate their relationships, we proposed a data-driven Bayesian Network (BN) model integrating physical information for risk analysis. Based on collected data, we combined prior knowledge with structure learning and parameter learning to obtain a BN model. In structure learning, we compared three structure learning algorithms including Bayesian search (BS), greedy thick thinning (GTT), and PC algorithm to obtain a robust directed acyclic graph (DAG). In parameter learning, we selected the expectation maximization (EM) algorithm to quantify the dependence and determine the probability distribution of node variables. This study provides a method to capture crucial factors and their interdependent relationships. To illustrate the applicability of the model, we developed a data-driven BN by taking the blowout accident as the case study. Eventually, we introduced vulnerability and resilience metrics for prioritizing risks through network propagation to conduct emergency plans and mitigation strategies.

A Bayesian network structural learning algorithm for calculating the failure probabilities of complex engineering systems with limited data

The failure of complex engineering systems is easy to lead to disastrous consequences. To prevent the failure, it is necessary to model complex engineering systems using probabilistic techniques with limited data which is a major feature of complex engineering systems. It is a good choice to perform such modeling using Bayesian network because of its advantages in probabilistic modeling. However, few Bayesian network structural learning algorithms are designed for complex engineering systems with limited data. Therefore, an algorithm for learning the Bayesian network structure of them should be developed. Based on the process of self-purification of water, a complex engineering system is segmented into three components according to the degree of difficulty in solving them. And then a Bayesian network learning algorithm with three components (TC), including PC algorithm, MIK algorithm which is originated by the paper through combining Mutual Information and K2 algorithm, and the Hill-Climbing method, is developed, i.e. TC algorithm. To verify its effectiveness, TC algorithm, K2 algorithm, and Max-Min Hill-Climbing are respectively used to learn Alarm network with different sizes of samples. The results imply that TC algorithm has the best performance. Finally, TC algorithm is applied to study tank spill accidents with 220 samples.

Automated hyperparameter selection for the PC algorithm

• The performance of the PC algorithm depends heavily on the choice of the Type I error rate α . • We propose to automatically tune α for a user chosen metric using a procedure called AutoPC. • AutoPC double checks the output of PC by rerunning the algorithm on restricting conditioning sets. • AutoPC provably maximizes the chosen metric and recovers the true CPDAG in the sample limit. • The algorithm also easily achieves state-of-the-art accuracy across multiple metrics with both synthetic and real data. The PC algorithm infers causal relations using conditional independence tests that require a pre-specified Type I α level. PC is however unsupervised, so we cannot tune α using traditional cross-validation. We therefore propose AutoPC, a fast procedure that optimizes α directly for a user chosen metric. We in particular force PC to double check its output by executing a second run on the recovered graph. We choose the final output as the one which maximizes stability between the two runs. AutoPC consistently outperforms the state of the art across multiple metrics.

Comparison of Myoelectric Control Schemes for Simultaneous Hand and Wrist Movement using Chronically Implanted Electromyography: A Case Series.

A current biomedical engineering challenge is the development of a system that allows fluid control of multi-functional prosthetic devices through a human-machine interface. Here we probe this challenge by studying two subjects with trans-radial limb loss as they control a virtual hand and wrist system using 6 or 8 chronically implanted intramuscular electromyographic (iEMG) signals. The subjects successfully controlled a 4, 5, and 6 Degrees of Freedom (DoF's) virtual hand and wrist systems to perform a target matching task. Two control systems were evaluated where one tied EMG features directly to movement directions (Direct Control) and the other method determines user intent in the context of prior training data (Linear Interpolation). Subjects successfully matched most targets with both controllers but differences were seen as the complexity of the virtual limb system increased. The Direct Control method encountered difficulty due to crosstalk at higher DoF's. The Linear Interpolation method reduced crosstalk effects and outperformed Direct Control at higher DoF's. This work also studied the use of the Postural Control Algorithm to control the hand postures simultaneously with wrist degrees of freedom. This work presents preliminary evidence that the PC algorithm can be used in conjunction with wrist control, that Direct Control with iEMG signals allows stable 4-DoF control, and that EMG pre-processing using the Linear Interpolation method can improve performance at 5 and 6-DoF's.

A general-purpose pipeline to interface the Tympan hardware with an external computer

Some audio applications require resource-heavy algorithms which cannot be run on real-time digital signal processors such as the Tympan hardware directly due to memory and processing constraints. These algorithms can, however, run on an external computer (PC), and their outcomes can be relayed back to the Tympan where necessary adjustments can be made in the audio processing elements of the underlying hardware. The proposed pipeline includes a 4-channel audio input, the algorithm running on the PC, the Tympan hardware, and a 4-channel output. The Tympan hardware acquires input audio from the Tympan’s onboard and/or external microphones with the I2S protocol. The input data are transmitted to the algorithm running on the external PC via USB/Serial communication by exposing the Tympan as a soundcard interface. The output of the PC algorithm is then sent back to the Tympan, through serial communication, so that the underlying audio processing parameters are adjusted. A possible use case is also discussed: a detection algorithm runs on the PC and serial commands are sent back to Tympan to alter the audio outputs, e.g., by playing masking noise. The pipeline is built such that each element can be used independently, abstracting the interconnection of the pipeline elements.

Risk spillover network structure learning for correlated financial assets: A directed acyclic graph approach

Using cross-asset return data in global financial markets, we propose a novel empirical framework to identify the causal structure of the asset risk spillover network. The joint return distribution of the global financial system can be characterized using a directed acyclic graph approach. However, since assets tend to be highly correlated during market turbulence, when adopting a nodewise penalized regression approach for neighborhood estimation, parameter estimates will receive large standard errors, and edges cannot be reliably estimated. In this work, we propose a two-stage approach for directed acyclic graph skeleton estimation for highly correlated variables. In the first stage, a variable screening ensemble is incorporated into the sparse partial least squares regression method to both reduce the size of the active variables set and impose an adaptive penalization on the weight vectors. In the second stage, a modified PC algorithm based on Gram-Schmidt orthogonalization is applied to remove the false positive edges. Simulation studies are conducted to demonstrate the effectiveness of the proposed method. Finally, we apply our method to analyze the asset risk spillover channels for international financial assets during the COVID-19 pandemic.

Constraint-based learning for non-parametric continuous bayesian networks

Modeling high-dimensional multivariate distributions is a computationally challenging task. In the discrete case, Bayesian networks have been successfully used to reduce the complexity and to simplify the problem. However, they lack of a general model for continuous variables. In order to overcome this problem, Elidan (2010) proposed the model of copula Bayesian networks that parametrizes Bayesian networks using copula functions. We propose a new learning algorithm for this model based on a PC algorithm and a conditional independence test proposed by Bouezmarni et al. (2009). This test being non-parametric, no model assumptions are made allowing it to be as general as possible. This algorithm is compared on generated data with the parametric method proposed by Elidan (2010) and proves to have better results.

A Dynamic Bayesian Network-Based Real-Time Crash Prediction Model for Urban Elevated Expressway

Traffic crash is a complex phenomenon that involves coupling interdependency among multiple influencing factors. Considering that interdependency is critical for predicting crash risk accurately and contributes to revealing the underlying mechanism of crash occurrence as well, the present study attempts to build a Real-Time Crash Prediction Model (RTCPM) for urban elevated expressway accounting for the dynamicity and coupling interdependency among traffic flow characteristics before crash occurrence and identify the most probable risk propagation path and the most significant contributors to crash risk. In this study, Dynamic Bayesian Network (DBN) was the framework of the RTCPM. Random Forest (RF) method was employed to identify the most important variables, which were used to build DBN-based RTCPMs. The PC algorithm combined with expert experience was further applied to investigate the coupling interdependency among traffic flow characteristics in the DBN model. A comparative analysis among the improved DBN-based RTCPM considering the interdependency, the original DBN-based RTCPM without considering the interdependency, and Multilayer Perceptron (MLP) was conducted. Besides, the sensitivity and strength of influences analyses were utilized to identify the most probable risk propagation path and the most significant contributors to crash risk. The results showed that the improved DBN-based RTCPM had better prediction performance than the original DBN-based RTCPM and the MLP based RTCPM. The most probable risk influencing path was identified as follows: speed on current segment (V) (time slice 2)⟶V (time slice 1)⟶speed on upstream segment (U_V) (time slice 1)⟶Traffic Performance Index (TPI) (time slice 1)⟶crash risk on current segment. The most sensitive contributor to crash risk in this path was V (time slice 2), followed by TPI (time slice 1), V (time slice 1), and U_V (time slice 1). These results indicate that the improved DBN-based RTCPM has the potential to predict crashes in real time for urban elevated expressway. Besides, it contributes to revealing the underlying mechanism of crash and formulating the real-time risk control measures.

MRPC: An R Package for Inference of Causal Graphs.

Understanding the causal relationships between variables is a central goal of many scientific inquiries. Causal relationships may be represented by directed edges in a graph (or equivalently, a network). In biology, for example, gene regulatory networks may be viewed as a type of causal networks, where X→Y represents gene X regulating (i.e., being causal to) gene Y. However, existing general-purpose graph inference methods often result in a high number of false edges, whereas current causal inference methods developed for observational data in genomics can handle only limited types of causal relationships. We present MRPC (a PC algorithm with the principle of Mendelian Randomization), an R package that learns causal graphs with improved accuracy over existing methods. Our algorithm builds on the powerful PC algorithm (named after its developers Peter Spirtes and Clark Glymour), a canonical algorithm in computer science for learning directed acyclic graphs. The improvements in MRPC result in increased accuracy in identifying v-structures (i.e., X→Y←Z), and robustness to how the nodes are arranged in the input data. In the special case of genomic data that contain genotypes and phenotypes (e.g., gene expression) at the individual level, MRPC incorporates the principle of Mendelian randomization as constraints on edge direction to help orient the edges. MRPC allows for inference of causal graphs not only for general purposes, but also for biomedical data where multiple types of data may be input to provide evidence for causality. The R package is available on CRAN and is a free open-source software package under a GPL (≥2) license.

Applying PC Algorithm and GES to Three Clinical Data Sets: Heart Disease, Diabetes, and Hepatitis

The goal of many sciences, including those related to the clinical domain, is to discover the generative model, that is, to understand how variables in the data take on their values. This goal cannot be addressed directly using approaches such as machine learning and deep learning, as such methods focus more on the association between input and output variables. In this paper, we aim to show to the readers an alternative approach, which can be a more appropriate method to target such aforesaid research goal. This approach is called causal modeling. We will first begin with some application examples of machine learning and deep learning on clinical data, and then show our applications of causal modeling to three clinical real-world data sets. This paper is projected to be a concise guideline for researchers to causal modeling, as well as to choose suitable approaches for problems of interest.

Small Scale Non-Invasive Imaging Using Magnetic Induction Tomography - Hardware Desig

This study is conducted to preliminary image the conductivity profile through the development of small scale non-invasive Magnetic Induction Tomography (MIT) system. It is proved that the Magnetic Induction Tomography interested in mapping the passive electrical properties of materials; conductivity (σ), permittivity (e) and permeability (µ) in both process and medical tomography. The system is realized by designing the functional ferrite-core coil sensors, electronic measurements circuits for excitation and receiving coil, data acquisition system for transferring the data to the PC and suitable image reconstruction algorithm for providing the conductivity distributions measurement. The important characteristic for excitation coil is the one that can maintain the stability the optimum sine wave frequency ranging from 400 kHz up to 10 MHz. The sine waves are fed to the excitation coil through the application of high current amplifier component respectively. In the experiments, the copper phantom represent as high conductivity material were placed into the region of interest. The initial 16 channel MIT consists of 8 excitation coil and 8 receiving coil stacked alternately. On the receiving circuit, the major problem is the weak secondary signal perturbation sensed by the receiving coil has been improved by placing the variable amplifier on each receiver. The enhancement of conductivity profile imaging has been made by using a common Linear Back Projection (LBP) algorithm. The measurement was done on single and dual arrangement of copper phantom aligns in random coordinate so that the sensitivity of the excitation and receiving coil sensor can be experimentally observed. The imaging’s results show that the hardware’s and algorithm used was capable to process the data captured at the receiver. The results obtained can be useful for further improvement and research towards magnetic induction tomography.

Constructing gene regulatory networks from microarray data using non-Gaussian pair-copula Bayesian networks.

Many biological and biomedical research areas such as drug design require analyzing the Gene Regulatory Networks (GRNs) to provide clear insight and understanding of the cellular processes in live cells. Under normality assumption for the genes, GRNs can be constructed by assessing the nonzero elements of the inverse covariance matrix. Nevertheless, such techniques are unable to deal with non-normality, multi-modality and heavy tailedness that are commonly seen in current massive genetic data. To relax this limitative constraint, one can apply copula function which is a multivariate cumulative distribution function with uniform marginal distribution. However, since the dependency structures of different pairs of genes in a multivariate problem are very different, the regular multivariate copula will not allow for the construction of an appropriate model. The solution to this problem is using Pair-Copula Constructions (PCCs) which are decompositions of a multivariate density into a cascade of bivariate copula, and therefore, assign different bivariate copula function for each local term. In fact, in this paper, we have constructed inverse covariance matrix based on the use of PCCs when the normality assumption can be moderately or severely violated for capturing a wide range of distributional features and complex dependency structure. To learn the non-Gaussian model for the considered GRN with non-Gaussian genomic data, we apply modified version of copula-based PC algorithm in which normality assumption of marginal densities is dropped. This paper also considers the Dynamic Time Warping (DTW) algorithm to determine the existence of a time delay relation between two genes. Breast cancer is one of the most common diseases in the world where GRN analysis of its subtypes is considerably important; Since by revealing the differences in the GRNs of these subtypes, new therapies and drugs can be found. The findings of our research are used to construct GRNs with high performance, for various subtypes of breast cancer rather than simply using previous models.

Leveraging Bayesian networks and information theory to learn risk factors for breast cancer metastasis

BackgroundEven though we have established a few risk factors for metastatic breast cancer (MBC) through epidemiologic studies, these risk factors have not proven to be effective in predicting an individual’s risk of developing metastasis. Therefore, identifying critical risk factors for MBC continues to be a major research imperative, and one which can lead to advances in breast cancer clinical care. The objective of this research is to leverage Bayesian Networks (BN) and information theory to identify key risk factors for breast cancer metastasis from data.MethodsWe develop the Markov Blanket and Interactive risk factor Learner (MBIL) algorithm, which learns single and interactive risk factors having a direct influence on a patient’s outcome. We evaluate the effectiveness of MBIL using simulated datasets, and compare MBIL with the BN learning algorithms Fast Greedy Search (FGS), PC algorithm (PC), and CPC algorithm (CPC). We apply MBIL to learn risk factors for 5 year breast cancer metastasis using a clinical dataset we curated. We evaluate the learned risk factors by consulting with breast cancer experts and literature. We further evaluate the effectiveness of MBIL at learning risk factors for breast cancer metastasis by comparing it to the BN learning algorithms Necessary Path Condition (NPC) and Greedy Equivalent Search (GES).ResultsThe averages of the Jaccard index for the simulated datasets containing 2000 records were 0.705, 0.272, 0.228, and 0.147 for MBIL, FGS, PC, and CPC respectively. MBIL, NPC, and GES all learned that grade and lymph_nodes_positive are direct risk factors for 5 year metastasis. Only MBIL and NPC found that surgical_margins is a direct risk factor. Only NPC found that invasive is a direct risk factor. MBIL learned that HER2 and ER interact to directly affect 5 year metastasis. Neither GES nor NPC learned that HER2 and ER are direct risk factors.DiscussionThe results involving simulated datasets indicated that MBIL can learn direct risk factors substantially better than standard Bayesian network learning algorithms. An application of MBIL to a real breast cancer dataset identified both single and interactive risk factors that directly influence breast cancer metastasis, which can be investigated further.

Robust Conditional Independence maps of single-voxel Magnetic Resonance Spectra to elucidate associations between brain tumours and metabolites.

The aim of the paper is two-fold. First, we show that structure finding with the PC algorithm can be inherently unstable and requires further operational constraints in order to consistently obtain models that are faithful to the data. We propose a methodology to stabilise the structure finding process, minimising both false positive and false negative error rates. This is demonstrated with synthetic data. Second, to apply the proposed structure finding methodology to a data set comprising single-voxel Magnetic Resonance Spectra of normal brain and three classes of brain tumours, to elucidate the associations between brain tumour types and a range of observed metabolites that are known to be relevant for their characterisation. The data set is bootstrapped in order to maximise the robustness of feature selection for nominated target variables. Specifically, Conditional Independence maps (CI-maps) built from the data and their derived Bayesian networks have been used. A Directed Acyclic Graph (DAG) is built from CI-maps, being a major challenge the minimization of errors in the graph structure. This work presents empirical evidence on how to reduce false positive errors via the False Discovery Rate, and how to identify appropriate parameter settings to improve the False Negative Reduction. In addition, several node ordering policies are investigated that transform the graph into a DAG. The obtained results show that ordering nodes by strength of mutual information can recover a representative DAG in a reasonable time, although a more accurate graph can be recovered using a random order of samples at the expense of increasing the computation time.

Estimating and Controlling the False Discovery Rate of the PC Algorithm Using Edge-specific P-Values

Many causal discovery algorithms infer graphical structure from observational data. The PC algorithm in particular estimates a completed partially directed acyclic graph (CPDAG), or an acyclic graph containing directed edges identifiable with conditional independence testing. However, few groups have investigated strategies for estimating and controlling the false discovery rate (FDR) of the edges in the CPDAG. In this article, we introduce PC with p-values (PC-p), a fast algorithm that robustly computes edge-specific p-values and then estimates and controls the FDR across the edges. PC-p specifically uses the p-values returned by many conditional independence (CI) tests to upper bound the p-values of more complex edge-specific hypothesis tests. The algorithm then estimates and controls the FDR using the bounded p-values and the Benjamini-Yekutieli FDR procedure. Modifications to the original PC algorithm also help PC-p accurately compute the upper bounds despite non-zero Type II error rates. Experiments show that PC-p yields more accurate FDR estimation and control across the edges in a variety of CPDAGs compared to alternative methods.

Stable Specification Search in Structural Equation Models with Latent Variables

In our previous study, we introduced stable specification search for cross-sectional data (S3C). It is an exploratory causal method that combines the concept of stability selection and multi-objective optimization to search for stable and parsimonious causal structures across the entire range of model complexities. S3C, however, is designed to model causal relations among observed variables. In this study, we extended S3C to S3C-Latent, to model linear causal relations between latent variables that are measured through observed proxies. We evaluated S3C-Latent on simulated data and compared the results to those of PC-MIMBuild, an extension of the PC algorithm, the state-of-the-art causal discovery method. The comparison shows that S3C-Latent achieved better performance. We also applied S3C-Latent to real-world data of children with attention deficit/hyperactivity disorder and data about measuring mental abilities among pupils. The results are consistent with those of previous studies.

Adversarial data poisoning attacks against the PC learning algorithm

ABSTRACTData integrity is a key component of effective Bayesian network structure learning algorithms, namely PC algorithm, design and use. Given the role that integrity of data plays in these outcomes, this research demonstrates the importance of data integrity as a key component in machine learning tools in order to emphasize the need for carefully considering data integrity during tool development and utilization. To meet this purpose, we study how an adversary could generate a desired network with the PC algorithm. Given a Bayesian network and a database generated by and a second Bayesian network, , which is equal to , except for a minor change like a missing link, a reversed link, or an additional link, we explore and analyze what is the minimal number of changes such as additions, deletions, substitutions to that lead to a database that, when given as input to PC algorithm, results in .

Learning Causal Biological Networks With the Principle of Mendelian Randomization.

Although large amounts of genomic data are available, it remains a challenge to reliably infer causal (i. e., regulatory) relationships among molecular phenotypes (such as gene expression), especially when multiple phenotypes are involved. We extend the interpretation of the Principle of Mendelian randomization (PMR) and present MRPC, a novel machine learning algorithm that incorporates the PMR in the PC algorithm, a classical algorithm for learning causal graphs in computer science. MRPC learns a causal biological network efficiently and robustly from integrating individual-level genotype and molecular phenotype data, in which directed edges indicate causal directions. We demonstrate through simulation that MRPC outperforms several popular general-purpose network inference methods and PMR-based methods. We apply MRPC to distinguish direct and indirect targets among multiple genes associated with expression quantitative trait loci. Our method is implemented in the R package MRPC, available on CRAN (https://cran.r-project.org/web/packages/MRPC/index.html).

CuPC: CUDA-Based Parallel PC Algorithm for Causal Structure Learning on GPU

The main goal in many fields in the empirical sciences is to discover causal relationships among a set of variables from observational data. PC algorithm is one of the promising solutions to learn underlying causal structure by performing a number of conditional independence tests. In this paper, we propose a novel GPU-based parallel algorithm, called cuPC, to execute an order-independent version of PC. The proposed solution has two variants, cuPC-E and cuPC-S, which parallelize PC in two different ways for multivariate normal distribution. Experimental results show the scalability of the proposed algorithms with respect to the number of variables, the number of samples, and different graph densities. For instance, in one of the most challenging datasets, the runtime is reduced from more than 11 hours to about 4 seconds. On average, cuPC-E and cuPC-S achieve 500 X and 1300 X speedup, respectively, compared to serial implementation on CPU. The source code of cuPC is available online [1].