Structure Learning Algorithm Research Articles

Revealing host genome–microbiome networks underlying feed efficiency in dairy cows

Ruminants have the ability to digest human-inedible plant materials, due to the symbiotic relationship with the rumen microbiota. Rumen microbes supply short chain fatty acids, amino acids, and vitamins to dairy cows that are used for maintenance, growth, and lactation functions. The main goal of this study was to investigate gene-microbiome networks underlying feed efficiency traits by integrating genotypic, microbial, and phenotypic data from lactating dairy cows. Data consisted of dry matter intake (DMI), net energy secreted in milk, and residual feed intake (RFI) records, SNP genotype, and 16S rRNA rumen microbial abundances from 448 mid-lactation Holstein cows. We first assessed marginal associations between genotypes and phenotypic and microbial traits through genomic scans, and then, in regions with multiple significant hits, we assessed gene-microbiome-phenotype networks using causal structural learning algorithms. We found significant regions co-localizing the rumen microbiome and feed efficiency traits. Interestingly, we found three types of network relationships: (1) the cow genome directly affects both rumen microbial abundances and feed efficiency traits; (2) the cow genome (Chr3: 116.5 Mb) indirectly affects RFI, mediated by the abundance of Syntrophococcus, Prevotella, and an unknown genus of Class Bacilli; and (3) the cow genome (Chr7: 52.8 Mb and Chr11: 6.1–6.2 Mb) affects the abundance of Rikenellaceae RC9 gut group mediated by DMI. Our findings shed light on how the host genome acts directly and indirectly on the rumen microbiome and feed efficiency traits and the potential benefits of the inclusion of specific microbes in selection indexes or as correlated traits in breeding programs. Overall, the multistep approach described here, combining whole-genome scans and causal network reconstruction, allows us to reveal the relationship between genome and microbiome underlying dairy cow feed efficiency.

Learning differentiable logic programs for abstract visual reasoning

AbstractVisual reasoning is essential for building intelligent agents that understand the world and perform problem-solving beyond perception. Differentiable forward reasoning has been developed to integrate reasoning with gradient-based machine learning paradigms. However, due to the memory intensity, most existing approaches do not bring the best of the expressivity of first-order logic, excluding a crucial ability to solve abstract visual reasoning, where agents need to perform reasoning by using analogies on abstract concepts in different scenarios. To overcome this problem, we propose NEUro-symbolic Message-pAssiNg reasoNer (NEUMANN), which is a graph-based differentiable forward reasoner, passing messages in a memory-efficient manner and handling structured programs with functors. Moreover, we propose a computationally-efficient structure learning algorithm to perform explanatory program induction on complex visual scenes. To evaluate, in addition to conventional visual reasoning tasks, we propose a new task, visual reasoning behind-the-scenes, where agents need to learn abstract programs and then answer queries by imagining scenes that are not observed. We empirically demonstrate that NEUMANN solves visual reasoning tasks efficiently, outperforming neural, symbolic, and neuro-symbolic baselines.

Soft sensor modeling method and application based on TSECIT2FNN-LSTM

To address the issue of low accuracy in soft sensor modeling of key variables caused by multi-variable coupling and parameter sensitivity in complex processes, this paper introduces a TSK-type-based self-evolving compensatory interval type-2 fuzzy Long short-term memory (LSTM) neural network (TSECIT2FNN-LSTM) soft sensor model. The proposed TSECIT2FNN-LSTM integrates the LSTM neural network with the interval type-2 fuzzy inference system to address long-term dependencies in sequence data by utilizing the gate mechanism of the LSTM neural network. The TSECIT2FNN-LSTM structure learning algorithm uses the firing strength of the network rule antecedent to decide whether to generate new rules to improve the rationality of the network structure. TSECIT2FNN-LSTM parameter learning utilizes the gradient descent method to optimize network parameters. However, unlike other interval type-2 fuzzy neural network gradient calculation processes, the error term in the LSTM node parameter gradient of TSECIT2FNN-LSTM is propagated backwards in the time dimension. Additionally, the error term is simultaneously transferred to the upper layer network to enhance network prediction accuracy and memory capabilities. The TSECIT2FNN-LSTM soft sensor model is utilized to predict the alcohol concentration in wine and the nitrogen oxide emission in gas turbines. Experimental results demonstrate that the proposed TSECIT2FNN-LSTM soft sensing model achieves higher prediction accuracy compared to other models.

Statistical Inference for Hüsler–Reiss Graphical Models Through Matrix Completions

The severity of multivariate extreme events is driven by the dependence between the largest marginal observations. The Hüsler–Reiss distribution is a versatile model for this extremal dependence, and it is usually parameterized by a variogram matrix. In order to represent conditional independence relations and obtain sparse parameterizations, we introduce the novel Hüsler–Reiss precision matrix. Similarly to the Gaussian case, this matrix appears naturally in density representations of the Hüsler–Reiss Pareto distribution and encodes the extremal graphical structure through its zero pattern. For a given, arbitrary graph we prove the existence and uniqueness of the completion of a partially specified Hüsler–Reiss variogram matrix so that its precision matrix has zeros on non-edges in the graph. Using suitable estimators for the parameters on the edges, our theory provides the first consistent estimator of graph structured Hüsler–Reiss distributions. If the graph is unknown, our method can be combined with recent structure learning algorithms to jointly infer the graph and the corresponding parameter matrix. Based on our methodology, we propose new tools for statistical inference of sparse Hüsler–Reiss models and illustrate them on large flight delay data in the United States, as well as Danube river flow data. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Adversarial Missingness Attacks on Causal Structure Learning

Causality-informed machine learning has been proposed as an avenue for achieving many of the goals of modern machine learning, from ensuring generalization under domain shifts to attaining fairness, robustness, and interpretability. A key component of causal machine learning is the inference of causal structures from observational data; in practice, this data may be incompletely observed. Prior work has demonstrated that adversarial perturbations of completely observed training data may be used to force the learning of inaccurate causal structural models (SCMs). However, when the data can be audited for correctness (e.g., it is cryptographically signed by its source), this adversarial mechanism is invalidated. This work introduces a novel attack methodology wherein the adversary deceptively omits a portion of the true training data to bias the learned causal structures in a desired manner (under strong signed sample input validation, this behavior seems to be the only strategy available to the adversary). Under this model, theoretically sound attack mechanisms are derived for the case of arbitrary SCMs, and a sample-efficient learning-based heuristic is given. Experimental validation of these approaches on real and synthetic data sets, across a range of SCMs from the family of additive noise models (linear Gaussian, linear non-Gaussian, and non-linear Gaussian) demonstrates the effectiveness of adversarial missingness attacks at deceiving popular causal structure learning algorithms.

A novel constraint-based structure learning algorithm using marginal causal prior knowledge

Causal discovery with prior knowledge is important for improving performance. We consider the incorporation of marginal causal relations, which correspond to the presence or absence of directed paths in a causal model. We propose the Marginal Prior Causal Knowledge PC (MPPC) algorithm to incorporate marginal causal relations into a constraint-based structure learning algorithm. We provide the theorems of conditional independence properties by combining observational data and marginal causal relations. We compare the MPPC algorithm with other structure learning methods in both simulation studies and real-world networks. The results indicate that, compare with other constraint-based structure learning methods, MPPC algorithm can incorporate marginal causal relations and is more effective and more efficient.

Efficient Local Coherent Structure Learning via Self-Evolution Bipartite Graph.

Dimensionality reduction (DR) targets to learn low-dimensional representations for improving discriminability of data, which is essential for many downstream machine learning tasks, such as image classification, information clustering, etc. Non-Gaussian issue as a long-standing challenge brings many obstacles to the applications of DR methods that established on Gaussian assumption. The mainstream way to address above issue is to explore the local structure of data via graph learning technique, the methods based on which however suffer from a common weakness, that is, exploring locality through pairwise points causes the optimal graph and subspace are difficult to be found, degrades the performance of downstream tasks, and also increases the computation complexity. In this article, we first propose a novel self-evolution bipartite graph (SEBG) that uses anchor points as the landmark of subclasses, and learns anchor-based rather than pairwise relationships for improving the efficiency of locality exploration. In addition, we develop an efficient local coherent structure learning (ELCS) algorithm based on SEBG, which possesses the ability of updating the edges of graph in learned subspace automatically. Finally, we also provide a multivariable iterative optimization algorithm to solve proposed problem with strict theoretical proofs. Extensive experiments have verified the superiorities of the proposed method compared to related SOTA methods in terms of performance and efficiency on several real-world benchmarks and large-scale image datasets with deep features.

Analyzing risk influencing factors of ship collision accidents: A data-driven Bayesian network model integrating physical knowledge

The interaction and propagation effects among risk-influencing factors (RIFs) often lead to maritime accidents. Therefore, to support maritime risk management in decision-making, it is imperative to conduct a quantitative risk assessment (QRA) of these accidents, which requires a methodology capable of capturing causal relationships in limited cases. To achieve this goal, this study proposes a data-driven Bayesian network (BN) model that integrates physical knowledge with the QRA. Based on the collected data, a combination of domain knowledge, structure learning, and parameter learning is employed to construct the model. In the data-driven phase, three structural learning algorithms including Bayesian search (BS), Greedy thick thinning (GTT), and Peter-Clark (PC) algorithms, were used in combination with the expectation maximization algorithm of parameter learning. Then, using four indices calculated by the confusion matrix, the most fitting algorithm for structure learning was chosen, and the confusion matrix was obtained by five-fold cross-validation. Finally, network propagation impact (NPI) is introduced to prioritize RIFs. Preventive measures and emergency plans are formulated based on the network resilience metric (NRM) and network vulnerability metric (NVM). This approach leverages the objectivity of a data-driven BN and integrates domain expertise to enhance model validity. A case study of collisions was conducted to demonstrate the applicability of the model. The data were sourced from 327 collision accident reports provided by the China Maritime Safety Administration, and the PC algorithm was chosen. Findings indicate that the “Management system”, “Supervision”, and “Crew training” RIFs are prioritized for preventive measures, whereas “Communication” receives more resources for emergency handling. The results suggest the formulation of risk management strategies for practitioners to improve maritime safety.

A data-driven Bayesian network of management and organizational factors for human reliability analysis in the process industry

According to historical statistical data, management and organizational factors (MOFs) contribute more to process accidents than technique factors. Under the umbrella of socio-tech system theory, human reliability analysis (HRA) has become a critical part of systemic probability risk analysis. In many HRA techniques, MOFs are among the performance shaping factors (PSFs). However, the interactions and causality of MOFs to human errors are still difficult to quantify and lack validation. To fill these gaps, a framework is proposed, considering data source selection, CBN construction algorithm comparison, and results validation. The case study employed the open access eMARS database as a data source. The optimized hybrid structure learning algorithm and Bayesian criteria parameter learning algorithm are employed to build a Causal Bayesian Network (CBN) of (MOFs) that lead to human error. The proposed kernel CBN is validated through prediction accuracy and sensitivity analysis. For theoretical contribution, the validated kernel BN could generally serve as the heart part of more specific CBNs as a basis for future works. For practical applications, an application shows the model's ability to quantify the contribution of MOFs to system reliability. The results show that human-machine interacting system reliability is most sensitive to organizational factors such as adequate training and procedures.

Programming knowledge tracing based on heterogeneous graph representation

Existing programming knowledge tracing methods have not deeply explored the relationships between knowledge concepts and programming questions or students’ codes, which leads to insufficient representation of programming questions and students’ codes and insufficient tracing of students’ knowledge states. To address these problems, this study proposes a programming knowledge tracing method based on a heterogeneous graph representation. We constructed a heterogeneous graph, where nodes represent the knowledge concept, programming question, and student code, and edges represent the relationships among knowledge concepts, programming questions, and student code. Subsequently, we leverage a graph structure learning algorithm to exploit the potential connections among them, aiming to enhance the graph structure information. Based on the graph, we further fully integrated the knowledge concept information into the representations of programming questions and student codes through the graph convolutional representation learning algorithm. Moreover, an attention mechanism is utilized to compute the effect of programming questions and student codes on the knowledge state and update the student's knowledge state in conjunction with the corresponding answer results. Furthermore, the proposed method provides an assessment of the mastery level for each knowledge concept rather than only assessing each question using traditional methods. Experiments on two datasets, CodeWorkout and CodeNet, show that the proposed method achieves an average improvement of 4.28% in ACC and 6.48% in AUC compared with the baseline models, validating the effectiveness of the proposed method.

The impact of variable ordering on Bayesian network structure learning

Causal Bayesian Networks (CBNs) provide an important tool for reasoning under uncertainty with potential application to many complex causal systems. Structure learning algorithms that can tell us something about the causal structure of these systems are becoming increasingly important. In the literature, the validity of these algorithms is often tested for sensitivity over varying sample sizes, hyper-parameters, and occasionally objective functions, but the effect of the order in which the variables are read from data is rarely quantified. We show that many commonly-used algorithms, both established and state-of-the-art, are more sensitive to variable ordering than these other factors when learning CBNs from discrete variables. This effect is strongest in hill-climbing and its variants where we explain how it arises, but extends to hybrid, and to a lesser-extent, constraint-based algorithms. Because the variable ordering is arbitrary, any significant effect it has on learnt graph accuracy is concerning, and raises questions about the validity of both many older and more recent results produced by these algorithms in practical applications and their rankings in performance evaluations.

Bayesian network structure learning with a new ensemble weights and edge constraints setting mechanism

Bayesian networks (BNs) are highly effective in handling uncertain problems, which can assist in decision-making by reasoning with limited and incomplete information. Learning a faithful directed acyclic graph (DAG) from a large number of complex samples of a joint distribution is currently a challenging combinatorial problem. Due to the growing volume and complexity of data, some Bayesian structure learning algorithms are ineffective and lack the necessary precision to meet the required needs. In this paper, we propose a new PCCL-CC algorithm. To ensure the accuracy of the network structure, we introduce the new ensemble weights and edge constraints setting mechanism. In this mechanism, we employ a method that estimates the interaction between network nodes from multiple perspectives and divides the learning process into multiple stages. We utilize an asymmetric weighted ensemble method and adaptively adjust the network structure. Additionally, we propose a causal discovery method that effectively utilizes the causal relationships among data samples to correct the network structure and mitigate the influence of Markov equivalence classes (MEC). Experimental results on real datasets demonstrate that our approach outperforms state-of-the-art methods.

Tuning structure learning algorithms with out-of-sample and resampling strategies

One of the challenges practitioners face when applying structure learning algorithms to their data involves determining a set of hyperparameters; otherwise, a set of hyperparameter defaults is assumed. The optimal hyperparameter configuration often depends on multiple factors, including the size and density of the usually unknown underlying true graph, the sample size of the input data, and the structure learning algorithm. We propose a novel hyperparameter tuning method, called the Out-of-sample Tuning for Structure Learning (OTSL), that employs out-of-sample and resampling strategies to estimate the optimal hyperparameter configuration for structure learning, given the input dataset and structure learning algorithm. Synthetic experiments show that employing OTSL to tune the hyperparameters of hybrid and score-based structure learning algorithms leads to improvements in graphical accuracy compared to the state-of-the-art. We also illustrate the applicability of this approach to real datasets from different disciplines.

Analysis of factors influencing the degree of accidental injury of bicycle riders considering data heterogeneity and imbalance.

Bicycle safety has emerged as a pressing concern within the vulnerable transportation community. Numerous studies have been conducted to identify the significant factors that contribute to the severity of cyclist injuries, yet the findings have been subject to uncertainty due to unobserved heterogeneity and class imbalance. This research aims to address these issues by developing a model to examine the impact of key factors on cyclist injury severity, accounting for data heterogeneity and imbalance. To incorporate unobserved heterogeneity, a total of 3,895 bicycle accidents were categorized into three homogeneous sub-accident clusters using Latent Class Cluster Analysis (LCA). Additionally, five over-sampling techniques were employed to mitigate the effects of data imbalance in each accident cluster category. Subsequently, Bayesian Network (BN) structure learning algorithms were utilized to construct 32 BN models after pairing the accident data from the four accident cluster types before and after sampling. The optimal BN models for each accident cluster type provided insights into the key factors associated with cyclist injury severity. The results indicate that the key factors influencing serious cyclist injuries vary heterogeneously across different accident clusters. Female cyclists, adverse weather conditions such as rain and snow, and off-peak periods were identified as key factors in several subclasses of accident clusters. Conversely, factors such as the week of the accident, characteristics of the trafficway, the season, drivers failing to yield to the right-of-way, distracted cyclists, and years of driving experience were found to be key factors in only one subcluster of accident clusters. Additionally, factors such as the time of the crash, gender of the cyclist, and weather conditions exhibit varying levels of heterogeneity across different accident clusters, and in some cases, exhibit opposing effects.

Construction of time series causal network based on partial rank correlation

Discovering causal relationships from time series data is a significant research area in data mining. To tackle the challenges of identifying and constructing causal networks in non-linear, high-dimensional time series data, we introduce the partial rank correlation coefficient and propose two new structure learning algorithms suitable for time-series causal network modeling. In this paper, we make three main contributions. Firstly, we demonstrate the suitability of the partial rank correlation as a criterion for independence testing. Secondly, by integrating the partial rank correlation with constraint-based causal discovery methods, we propose the TS-PRCS algorithm for temporal causal network discovery. Thirdly, we combine ideas from local learning, constraint-based methods, and score-based search to propose the TS-MCHC algorithm, which combines mixed constraints and hill-climbing search for time-series causal discovery. Finally, we validate the effectiveness of the TS-PRCS and TS-MCHC algorithms through experiments on both simulated and real-world data. Compared to existing methods, our proposed algorithms exhibit robust causal detection capabilities in diverse non-linear environments and demonstrate efficient performance in high-dimensional time series networks. The experimental results affirm the ability of our methods to effectively uncover causal relationships among time series data, presenting promising prospects for practical applications.

Modeling the causal mechanism in process safety management (PSM) systems from historical accidents

Process safety management (PSM) underpins reducing major accidents in the process industries. For decades, PSM systems have been implemented globally. However, few studies have modeled the complexities within the PSM system, leading to some underlying chronic problems within organizations that go unnoticed. Here, we investigate causal mechanisms among PSM elements from historical accidents to deliver a practical tool for safety intervention formulation. Specifically, based on the Center for Chemical Process Safety's (CCPS) PSM framework, 100 chemical accident reports derived from the U.S. Chemical Safety and Hazard Investigation Board (CSB) were analyzed and converted into structured information stored in 22 PSM elements. Then, employing two causal structure learning algorithms, the causal structures among 22 elements were learned from the processed accident information. Finally, the causal effects among PSM elements were estimated by extending Rubin's counterfactual causal framework to an approach applicable to multi-treatment scenarios. We demonstrate that the causal structures and effects learned from accident reports reveal the causal mechanism within the PSM system. A total of 20 significant causal relationships among single elements, nine significant causal relationships among element combinations, and several interesting PSM failure paths were identified. According to these causal patterns, the PSM effort will shift from the traditional "what to do" to "how to do it."

Predictive coding with spiking neurons and feedforward gist signaling.

Predictive coding (PC) is an influential theory in neuroscience, which suggests the existence of a cortical architecture that is constantly generating and updating predictive representations of sensory inputs. Owing to its hierarchical and generative nature, PC has inspired many computational models of perception in the literature. However, the biological plausibility of existing models has not been sufficiently explored due to their use of artificial neurons that approximate neural activity with firing rates in the continuous time domain and propagate signals synchronously. Therefore, we developed a spiking neural network for predictive coding (SNN-PC), in which neurons communicate using event-driven and asynchronous spikes. Adopting the hierarchical structure and Hebbian learning algorithms from previous PC neural network models, SNN-PC introduces two novel features: (1) a fast feedforward sweep from the input to higher areas, which generates a spatially reduced and abstract representation of input (i.e., a neural code for the gist of a scene) and provides a neurobiological alternative to an arbitrary choice of priors; and (2) a separation of positive and negative error-computing neurons, which counters the biological implausibility of a bi-directional error neuron with a very high baseline firing rate. After training with the MNIST handwritten digit dataset, SNN-PC developed hierarchical internal representations and was able to reconstruct samples it had not seen during training. SNN-PC suggests biologically plausible mechanisms by which the brain may perform perceptual inference and learning in an unsupervised manner. In addition, it may be used in neuromorphic applications that can utilize its energy-efficient, event-driven, local learning, and parallel information processing nature.

Finding community structure in Bayesian networks by heuristic K-standard deviation method

When constructing a Bayesian network for high-dimensional data, due to the complex relationships among distinct nodes, the difficulty in detecting the community structure will directly restrict the feasibility of the divide-and-conquer learning algorithm. This study attempts to solve this problem from the shortest path perspective and proposes the heuristic K-standard deviation algorithm. Firstly, we design a novel heuristic function and drive the A* algorithm to find the shortest paths between nodes in the Bayesian network. In addition, the new heuristic function is theoretically proved to be admissible and consistent. Experiments on different synthetic datasets and benchmark Bayesian networks verify that the proposed heuristic K-standard deviation algorithm generally gets better clustering performance than other representative algorithms and improves the efficiency and accuracy of the conventional Bayesian network structure learning algorithms, especially for high-dimensional data.

Delving into Causal Discovery in Health-Related Quality of Life Questionnaires

Questionnaires on health-related quality of life (HRQoL) play a crucial role in managing patients by revealing insights into physical, psychological, lifestyle, and social factors affecting well-being. A methodological aspect that has not been adequately explored yet, and is of considerable potential, is causal discovery. This study explored causal discovery techniques within HRQoL, assessed various considerations for reliable estimation, and proposed means for interpreting outcomes. Five causal structure learning algorithms were employed to examine different aspects in structure estimation based on simulated data derived from HRQoL-related directed acyclic graphs. The performance of the algorithms was assessed based on various measures related to the differences between the true and estimated structures. Moreover, the Resource Description Framework was adopted to represent the responses to the HRQoL questionnaires and the detected cause–effect relationships among the questions, resulting in semantic knowledge graphs which are structured representations of interconnected information. It was found that the structure estimation was impacted negatively by the structure’s complexity and favorably by increasing the sample size. The performance of the algorithms over increasing sample size exhibited a similar pattern, with distinct differences being observed for small samples. This study illustrates the dynamics of causal discovery in HRQoL-related research, highlights aspects that should be addressed in estimation, and fosters the shareability and interoperability of the output based on globally established standards. Thus, it provides critical insights in this context, further promoting the critical role of HRQoL questionnaires in advancing patient-centered care and management.

Assessment of the organizational factors in incident management practices in healthcare: A tree augmented Naive Bayes model.

Despite the exponential transformation occurring in the healthcare industry, operational failures pose significant challenges in the delivery of safe and efficient care. Incident management plays a crucial role in mitigating these challenges; however, it encounters limitations due to organizational factors within complex and dynamic healthcare systems. Further, there are limited studies examining the interdependencies and relative importance of these factors in the context of incident management practices. To address this gap, this study utilized aggregate-level hospital data to explore the influence of organizational factors on incident management practices. Employing a Bayesian Belief Network (BBN) structural learning algorithm, Tree Augmented Naive (TAN), this study assessed the probabilistic relationships, represented graphically, between organizational factors and incident management. Significantly, the model highlighted the critical roles of morale and staff engagement in influencing incident management practices within organizations. This study enhances our understanding of the importance of organizational factors in incident management, providing valuable insights for healthcare managers to effectively prioritize and allocate resources for continuous quality improvement efforts.