Bayesian Network Classifiers Research Articles

Usage of intellectual analysis of ecological-economic data in the pension insurance system and for forecasting expenditures on social protection and social security

The paper is devoted to an actual scientific and applied problem – the development of methodology for applying mathematical models and data mining methods for actuarial calculations in mandatory state pension insurance system. The paper describes methodology for modeling changes in the number of pension recipients taking into account the impact of environmental factors, in particular air pollution. The basis of the proposed method is a multi-model approach, characterized by combination of data mining and probabilistic models in the form of Bayesian network, which are appropriate in conditions of statistical, parametric and structural uncertainty.The proposed approach describes the change in number of pension recipients, in particular for disability and breadwinner loss, under influence of air pollution from organic and inorganic compounds. The scientific novelty of the paper is in the use of an ensemble of models including probabilistic and statistical models in the form of Bayesian network and regression models, in the system of actuarial calculations of mandatory state pension insurance.The paper considers several scenarios for the impact of pollutants on the growth of number of pension recipients. The indicator of the share of expenditures on social protection and social security of the population in the gross national product was chosen as the target variable of the process under study. Mathematical models were found to be adequate to the modeling process, and the Bayesian network classification error is about 20%. The model structure is built in Genie 2.0 modeling system. The principal component analysis, is used to reduce the data dimension. The proposed methodology can also be applied to other tasks of forecasting social protection and social security expenditures.

Retweeting behavior prediction based on dynamic Bayesian network classifier in microblogging networks

Nowadays, predicting user retweeting behavior in microblogging networks has gained considerable attention. Solving this problem allows for understanding the underlying mechanism of information diffusion and analyzing diffusion's evolution concerning a particular original microblog over time. Time-series data from microblogging networks are ubiquitous, and user retweeting behavior prediction is primarily considered a classification task. Consequently, there is a significant demand for accurate classification of time-series data. Several influential factors affect user retweeting behavior prediction in microblogging networks. However, much of the existing research focuses on studies that neglect the impact of users' indirect social influence strength. This neglect leads to inaccurate forecasts of indirect retweeting behavior for a particular original microblog. To address this, this paper presents a new time-sensitive measure of social influence strength based on a combination of users' temporal behavior patterns and the shortest cascade path length. Second, it proposes a prediction model based on Dynamic Bayesian Network derivative classifiers called V-DBNC to accurately predict user retweeting behavior over time. The proposed time-sensitive social influence strength factor and other factors, such as individual behavior and similarity-based driving factors, are applied to the V-DBNC model to achieve accurate user retweeting behavior prediction in the microblogging network. The joint density of factors in this classifier is estimated using a multivariate Gaussian kernel function with smoothing parameters. The V-DBNC model is empirically optimized by splitting the smoothing parameters into different-scale intervals and using the model averaging method to select the optimal classifier. Additionally, the proposed model's relatively simple structure helps overcome the overfitting problem and allows it to accumulate classification information through iterative evolution, promoting generalization. The real Twitter microblogging dataset is used to evaluate the performance of V-DBNC. The experimental results demonstrated that the proposed model outperforms other compared approaches when dealing with the classification of time-series data.

#1756 Development and internal validation of machine learning algorithms for mortality prediction model of people with DM and CKD

Abstract Background and Aims Diabetes Mellitus (DM) and Chronic Kidney Disease (CKD) are leading causes of morbidity and mortality, presenting challenges in patient management. The aim of this study is to develop a machine learning-based predictive model for mortality in DM and CKD patients, improving early intervention and treatment personalization. Method 3,637 participants with DM and CKD from the National Health and Nutrition Examination Survey (NHANES) from 1999 to 2018 were included. All-cause mortality was ascertained by linkage to National Death Index records through 31 December 2019 in NHANES. The dataset included clinical profiles, demographic details, and laboratory results of patients with DM and CKD. We performed 100 random groupings, dividing the data into a 75% training set and a 25% validation set each time. We then visualized the results of 100 AUC measurements using a boxplot. The objective was to forecast patient survival over 1, 3, 5, and 10-year horizons using a variety of machine learning algorithms. The algorithms tested included Logistic Regression (LR), Decision Tree (DT), Random Forest (RF), K-Nearest Neighbors (KNN), Gaussian Naive Bayes (GNB), Multinomial Naive Bayes (MNB), Bayesian Network Classifier (BNC), AdaBoost (Ada), Gradient Boosting (Gradient), and Extreme Gradient Boosting (XG). The performance of each algorithm was evaluated based on the Area Under the Receiver Operating Characteristic (ROC) Curve (AUC), with the results visualized in boxplot format. Results Upon result shown as the boxplots, it becomes evident that the Random Forest (RF) algorithm consistently exhibits one of the highest AUCs on average across all prediction intervals, suggesting a strong and stable predictive capacity. In the 1-year survival prediction, the RF algorithm demonstrates a highest AUC, implying consistent performance. The 3 and 5-year predictions reveal a slight dip in AUC for most models, including RF, but it still remains one of the top-performing algorithms with fewer outliers than XG and Gradient Boosting. The mean AUC of RF is particularly noteworthy at the 10-year mark, where most algorithms struggle with prediction accuracy, as indicated by lower median AUCs and greater variability. Notably, the RF algorithm identified six major factors for 1-year survival prediction: systolic blood pressure, uric acid, blood urea nitrogen, lactate dehydrogenase, lymphocyte count and total cholesterol, as well as three major factors for 10-year survival prediction: age, lymphocyte and gamma-glutamyl transferase. Conclusion Overall, the study indicates that while machine learning can be a powerful tool for survival prediction in patients with DM and CKD, the choice of algorithm is crucial, with RF standing out for its consistent and reliable performance.

ACME: A Classification Model for Explaining the Risk of Preeclampsia Based on Bayesian Network Classifiers and a Non-Redundant Feature Selection Approach

While preeclampsia is the leading cause of maternal death in Guayas province (Ecuador), its causes have not yet been studied in depth. The objective of this research is to build a Bayesian network classifier to diagnose cases of preeclampsia while facilitating the understanding of the causes that generate this disease. Data for the years 2017 through 2023 were gathered retrospectively from medical histories of patients treated at “IESS Los Ceibos” hospital in Guayaquil, Ecuador. Naïve Bayes (NB), The Chow–Liu Tree-Augmented Naïve Bayes (TANcl), and Semi Naïve Bayes (FSSJ) algorithms have been considered for building explainable classification models. A proposed Non-Redundant Feature Selection approach (NoReFS) is proposed to perform the feature selection task. The model trained with the TANcl and NoReFS was the best of them, with an accuracy close to 90%. According to the best model, patients whose age is above 35 years, have a severe vaginal infection, live in a rural area, use tobacco, have a family history of diabetes, and have had a personal history of hypertension are those with a high risk of developing preeclampsia.

Learning Bayesian Network Classifiers to Minimize the Class Variable Parameters

This study proposes and evaluates a new Bayesian network classifier (BNC) having an I-map structure with the fewest class variable parameters among all structures for which the class variable has no parent. Moreover, a new learning algorithm to learn our proposed model is presented. The proposed method is guaranteed to obtain the true classification probability asymptotically. Moreover, the method has lower computational costs than those of exact learning BNC using marginal likelihood. Comparison experiments have demonstrated the superior performance of the proposed method.

A Fast Multi-Network K-Dependence Bayesian Classifier for Continuous Features

One of the Bayesian network classifiers widely used in the classification is K-dependence Bayesian (KDB). However, most of the KDB classifiers build a single network on a class variable without considering dependencies between features in each class. Moreover, many KDB classifiers need the discretization process to handle continuous features. This paper aims to propose a fast Multi-Network K-Dependence Bayesian (MNKDB) classifier for continuous features. According to this aim, we propose a non-parametric approach that efficiently identifies dependencies between continuous features in each class with a low computational cost and without discretizing continuous features. The results indicate that the MNKDB classifier is more accurate than the state-of-the-art KDB classifiers, especially for datasets with more than three classes. The MNKDB classifier not only decreases the classification time but also deals with continuous variables without discretizing them. The results for K=2 show that the MNKDB classifier is 36.5, 31.8, and 14.2 times faster and 4.13%, 5.15%, and 5.48% more accurate than the state-of-the-art FKDB (Flexible KDB), KMM-KDB (Kernel Mixture Model based on KDB), and SKDB (Scalable KDB) classifiers, respectively.

Efficient heuristics for learning scalable Bayesian network classifier from labeled and unlabeled data

Efficient heuristics for learning scalable Bayesian network classifier from labeled and unlabeled data

High-Dimensional Bayesian Network Classification with Network Global-Local Shrinkage Priors

This article proposes a novel Bayesian binary classification framework for networks with labeled nodes. Our approach is motivated by applications in brain connectome studies, where the overarching goal is to identify both regions of interest (ROIs) in the brain and connections between ROIs that influence how study subjects are classified. We propose a novel binary logistic regression framework with the network as the predictor, and model the associated network coefficient using a novel class of global-local network shrinkage priors. We perform a theoretical analysis of a member of this class of priors (which we call the Network Lasso Prior) and show asymptotically correct classification of networks even when the number of network edges grows faster than the sample size. Two representative members from this class of priors, the Network Lasso prior and the Network Horseshoe prior, are implemented using an efficient Markov Chain Monte Carlo algorithm, and empirically evaluated through simulation studies and the analysis of a real brain connectome dataset.

Learning bayesian multinets from labeled and unlabeled data for knowledge representation

The Bayesian network classifiers (BNCs) learned from labeled training data are expected to generalize to fit unlabeled testing data based on the independent and identically distributed (i.i.d.) assumption, whereas the asymmetric independence assertion demonstrates the uncertainty of significance of dependency or independency relationships mined from data. A highly scalable BNC should form a distinct decision boundary that can be especially tailored to specific testing instance for knowledge representation. To address the issue of asymmetric independence assertion, in this paper we propose to learn k-dependence Bayesian multinet classifiers in the framework of multistage classification. By partitioning training set and pseudo training set according to high-confidence class labels, the dependency or independency relationships can be fully mined and represented in the topologies of the committee members. Extensive experimental results indicate that the proposed algorithm achieves competitive classification performance compared to single-topology BNCs (e.g., CFWNB, AIWNB and SKDB) and ensemble BNCs (e.g., WATAN, SA2DE, ATODE and SLB) in terms of zero-one loss, root mean square error (RMSE), Friedman test and Nemenyi test.

Framework and Change Path Analysis for the Improvement of Accountants’ Data Analysis Capability in the Era of “Digitization”

Abstract This paper first analyzes the importance of data analysis skills and uses information mining to discover valuable accounting knowledge. Secondly, in the process of calculating the weights of indicators, the Bayesian network classifier is used to enhance the data mining technology, filtering and processing the interfering information and redundant information to ensure the effectiveness of professional competence data acquisition. Finally, the framework of professional competence of accounting talents was constructed, the path of accountants’ change was analyzed, and the design of the numerical intelligence curriculum system was completed. The results show that the rating of data analysis ability is 4.63, and the average score of the others is roughly around 4, which proves the feasibility of the Numerical Intelligence Curriculum System.

Learning high-dependence Bayesian network classifier with robust topology

The increase in data variability and quantity makes it urgent for learning complex multivariate probability distributions. The state-of-the-art Tree Augmented Naive Bayes (TAN) classifier uses maximum weighted spanning tree (MWST) to graphically model data with excellent time and space complexity. In this paper, we theoretically prove the feasibility of scaling up one-dependence MWST to model high-dependence relationships, and then propose to apply a heuristic search strategy to improve the fitness of extended topology to data. The newly added edges to each attribute node may provide a local optimal solution. Then ensemble learning is introduced to improve the generalization performance and reduce the sensitivity to variation in training data. The experimental results on 32 benchmark datasets reveal that this highly scalable algorithm inherits the expressive power of TAN and achieves an excellent bias–variance tradeoff, and it also demonstrates competitive classification performance when compared to a range of high-dependence or ensemble learning algorithms.

A novel diagnosis method for schizophrenia based on globus pallidus data

This research aims to diagnose schizophrenia with machine learning-based algorithms. Bayesian neural network, logistic regression, decision tree, k-nearest neighbor, and gaussian kernel classification techniques are investigated to diagnose schizophrenia with data from 125 persons. This study showed that left lateral ventricles and left globus pallidus volumes and their percentages in the brain were significantly lower than HCs in FEP patients. Using brain volumes, we were able to diagnose FEP with an accuracy of 73.6 % via logistic regression and with an accuracy of 86.4 % using the SVM kernel classifier method. Therefore, brain volumes can be used to diagnose FEP with the SVM kernel classifier method.

Investigation of Ensemble Learning Method in Predicting Early Xerostomia Using Radiomics Features from CT and MRI

There are different side effects in radiotherapy of head and neck cancer including xerostomia. Xerostomia is due to irradiation of salivary glands during treatment. In the present study, the main aim is evaluation and validation of individual and ensemble classifiers for prediction of early xerostomia in radiotherapy of head and neck cancer. Totally 62 patients diagnosed with head and neck cancer were evaluated prospectively. For this purpose, patients' demographic information, dosimetric data, and the extracted features from CT and T1 and T2 MR images before radiotherapy treatments were used as input for the predictive model. There were two main steps in this study: 1- Comparison of the features from T1 and T2 MR images for prediction of xerostomia, and 2- Evaluation of the performance of the radiomics models using individual classifiers and combination of the classifiers using ensemble learning. Feature extraction was performed using 3D Slicer software and for both parotids, 642 features were extracted from both CT, and MRI and Pearson statistical test was used for feature selection. For model development, four models were used consist of Random Tree (RT), Neural Network (NN), Linear Support Vector Machine (LSVM) and Bayesian Network (BS) classifiers. Different algorithms combined by using the ensemble learning method. For evaluation of the performance of the constructed models, sensitivity, specificity, area under curve (AUC), and receiver operating characteristic (ROC) curve were utilized. The results of the present study show that the extracted features from T1 weighted images have superior prediction ability compared to the T2 weighted ones. The RT and LSVM models which were based on T1 weighted images have better performance than those with T2 weighted images. The AUC in the test group for these models are 0.95 and 0.76, respectively, while for the T2 weighted images the corresponding values are 0.75 and 0.63. The AUC for the test group for the RT-BN, RT-LSVM-BN and RT-NN-LSVM-BN models amount to 0.96, 0.91, and 0.88, respectively. The results show that the radiomics features from images before radiotherapy can be used as personalized and unique biomarkers for prediction of xerostomia. Ensemble classifiers can be more efficient than individual classifiers in prediction of early xerostomia.

MEASURING VULNERABILITY TO MULTIDIMENSIONAL POVERTY IN LATIN AMERICA

AbstractIn this paper, we perform estimates of vulnerability to multidimensional poverty for 17 Latin American countries at three points of time: 2005/2006, 2012, and 2017. We use a Multidimensional Bayesian Network Classifier model to estimate the conditional probability of being multidimensionally poor and then we use these probabilities and the standard downside semi‐deviation as the risk parameter to identify the vulnerable households. Despite significant reductions over the study period, in 2017 approximately 150 million people—excluding Guatemala, Nicaragua and Venezuela for which we do not have recent data—remained vulnerable to multidimensional poverty. We also observe that vulnerability to poverty is reduced at a much slower rate than poverty itself, revealing that achievements in SDG1 can be quite fragile. We perform a decomposition and find that as poverty decreases, risk‐induced vulnerability becomes relatively more important than poverty‐induced vulnerability. However, the poor‐vulnerable still constitute the core vulnerability group.

Exploiting the implicit independence assumption for learning directed graphical models

Bayesian network classifiers (BNCs) provide a sound formalism for representing probabilistic knowledge and reasoning with uncertainty. Explicit independence assumptions can effectively and efficiently reduce the size of the search space for solving the NP-complete problem of structure learning. Strong conditional dependencies, when added to the network topology of BNC, can relax the independence assumptions, whereas the weak ones may result in biased estimates of conditional probability and degradation in generalization performance. In this paper, we propose an extension to the k-dependence Bayesian classifier (KDB) that achieves the bias/variance trade-off by verifying the rationality of implicit independence assumptions implicated. The informational and probabilistic dependency relationships represented in the learned robust topologies will be more appropriate for fitting labeled and unlabeled data, respectively. The comprehensive experimental results on 40 UCI datasets show that our proposed algorithm achieves competitive classification performance when compared to state-of-the-art BNC learners and their efficient variants in terms of zero-one loss, root mean square error (RMSE), bias and variance.

Searching for dark matter subhalos in the Fermi-LAT catalog with Bayesian neural networks

About a third of the γ-ray sources detected by the Fermi Large Area Telescope (Fermi-LAT) remain unidentified, and some of these could be exotic objects such as dark matter subhalos. We present a search for these sources using Bayesian neural network classification methods applied to the latest 4FGL-DR3 Fermi-LAT catalog. We first simulate the γ-ray properties of dark matter subhalos using models from N-body simulations and semi-analytical approaches to the subhalo distribution. We then assess the detectability of this sample in the 4FGL-DR3 catalog using the Fermi-LAT analysis tools. We train our Bayesian neural network to identify candidate dark matter subhalos among the unidentified sources in the 4FGL-DR3 catalog. Our results allow us to derive conservative bounds on the dark matter annihilation cross section by excluding unidentified sources classified as astrophysical-like by our networks. We estimate the number of candidate dark matter subhalos for different dark matter masses and provide a publicly available list for further investigation. Our bounds on the dark matter annihilation cross section are comparable to previous results and become particularly competitive at high dark matter masses.

Risk prediction for petroleum exploration based on Bayesian network classifier

Effective prediction of petroleum exploration risks is critical for boosting oil and gas extraction efficiency and increasing economic benefits. To improve the accuracy and efficiency of risk prediction, this paper proposes a scalable k-dependence Bayesian network classifier (SKDB) method for solving the risk prediction problem of oil and gas exploration. The effectiveness and efficiency of the SKDB method were illustrated by taking the Jurassic Sangonghe Formation hydrocarbon exploration risk prediction in the Junggar Basin as an example. The model accuracy comparison results showed that SKDB has the highest accuracy compared with the state-of-the-art methods, and the accuracy was improved by 3.55∼7.63%. The oil and gas probability map prediction results indicated that the SKDB method not only showed high hydrocarbon probability in the reservoir areas, but also had better trend inference ability compared with other methods. Based on the SKDB prediction results, the remaining hydrocarbon favorable areas of the Sangonghe Formation reservoir were visualized and three types of favorable hydrocarbon distribution zones were preferentially selected, namely Type I (highly favorable), Type II (moderately favorable) and Type III (low favorable). The application results can provide a decision basis for the next oil and gas exploration and optimization strategy.

Multi-dimensional Bayesian network classifiers for partial label ranking

The label ranking problem consists in learning preference models from training datasets labeled with (possibly incomplete) rankings of the class labels. The goal is then to predict a ranking for a given unlabeled instance. This work focuses on a more general interpretation where both the training dataset and the prediction given as output allow tied class labels, i.e., there is no particular preference between them. This problem is known as the partial label ranking problem. This paper tackles the partial label ranking problem by transforming the ranking with ties into a set of discrete variables representing the preference relations (ranked ahead, tied with, and ranked behind) between each pair of class labels. The posterior probabilities for each pair are then used to fill the values of a preference matrix. This preference matrix is the basis for solving the rank aggregation problem required to obtain the output ranking with ties. This paper aims to exploit the resemblance of this problem with multi-label and multi-dimensional classification by studying the use of Bayesian network classifiers to compute the posterior probabilities for the new class structure, i.e., pairs of class labels. In particular, binary relevance with naive Bayes and averaged one-dependence estimators between the new class structure are used to solve the partial label ranking problem. Furthermore, bivariate relationships between all the pairs of class labels are considered. However, the complexity of the model grows significantly, which makes it necessary to reduce the number of allowed bivariate relationships between pairs. Thus, a feature selection method is included to select the more relevant subset of bivariate relationships. The experimental evaluation shows that our proposals are competitive in accuracy with the current instance-based and decision tree induction algorithms. Moreover, they outperform the existing mixture-based probabilistic graphical models, while the algorithms proposed are much faster.

Constraint-based and hybrid structure learning of multidimensional continuous-time Bayesian network classifiers

Learning the structure of continuous-time Bayesian networks directly from data has traditionally been performed using score-based structure learning algorithms. Only recently has a constraint-based method been proposed, proving to be more suitable under specific settings, as in modelling systems with variables having more than two states. As a result, studying diverse structure learning algorithms is essential to learn the most appropriate models according to data characteristics and task-related priorities, such as learning speed or accuracy. This article proposes alternative algorithms for learning multidimensional continuous-time Bayesian network classifiers, introducing, for the first time, constraint-based and hybrid algorithms for these models. Nevertheless, these contributions also apply to the simpler one-dimensional classification problem for which only score-based solutions exist in the literature. More specifically, the aforementioned constraint-based structure learning algorithm is first adapted to the supervised classification setting. Then, a novel algorithm of this kind, specifically tailored for the multidimensional classification problem, is presented to improve the learning times for the induction of multidimensional classifiers. Finally, a hybrid algorithm is introduced, attempting to combine the strengths of the score- and constraint-based approaches. Experiments with synthetic and real-world data are performed not only to validate the capabilities of the proposed algorithms but also to conduct a comparative study of the available competitors.

Weighted Bayesian network for the classification of unbalanced food safety data: Case study of risk-based monitoring of heavy metals.

Historical data on food safety monitoring often serve as an information source in designing monitoring plans. However, such data are often unbalanced: a small fraction of the dataset refers to food safety hazards that are present in high concentrations (representing commodity batches with a high risk of being contaminated, the positives) and a high fraction of the dataset refers to food safety hazards that are present in low concentrations (representing commodity batches with a low risk of being contaminated, the negatives). Such unbalanced datasets complicate modeling to predict the probability of contamination of commodity batches. This study proposes a weighted Bayesian network (WBN) classifier to improve the model prediction accuracy for the presence of food and feed safety hazards using unbalanced monitoring data, specifically for the presence of heavy metals in feed. Applying different weight values resulted in different classification accuracies for each involved class; the optimal weight value was defined as the value that yielded the most effective monitoring plan, that is, identifying the highest percentage of contaminated feed batches. Results showed that the Bayesian network classifier resulted in a large difference between the classification accuracy of positive samples (20%) and negative samples (99%). With the WBN approach, the classification accuracy of positive samples and negative samples were both around 80%, and the monitoring effectiveness increased from 31% to 80% for pre-set sample size of 3000. Results of this study can be used to improve the effectiveness of monitoring various food safety hazards in food and feed.