Bayesian Network Classifiers Research Articles

Flexible model weighting for one-dependence estimators based on point-wise independence analysis

Recent studies have shown that Bayesian network classifiers (BNCs) are powerful tools for knowledge representation and classification, and averaged one-dependence estimators (AODE) is one of the most popular and effective BNCs since it can achieve the tradeoff between bias and variance due to its independence assumptions and ensemble learning strategy. However, unverified independence assumptions may result in biased estimates of probability distribution and then degradation in classification performance. In this paper, we prove theoretically the uncertainty of probability-theoretic independence and propose to measure the independence between attribute values implicated in specific instance. The estimates of conditional probability can be finely tuned based on point-wise independence analysis. Point-wise log likelihood function is then applied as weighting metric for committee members of AODE to improve the estimate of joint probability. Extensive experiments on 36 benchmark datasets show that, compared to other state-of-the-art classifiers, weighted one-dependence estimators using point-wise independence analysis can achieve competitive classification performance in terms of zero-one loss, RMSE, bias-variance decomposition and conditional log likelihood.

Enhancement of Online Education in Engineering College Based on Mobile Wireless Communication Networks and IOT

The field of Engineering is that which needs a high level of analytical thinking, intuitive knowledge, and technical know-how. The area of communication engineering deals with different components including, wireless mobile services, radio, broadband, web and satellites. There is a rapid decline in the quality of students produced by engineering faculties as a result of sufficient and quality methods and frameworks of student assessment. The production of high-potential engineers is limited by the utilization of old and traditional education methodology and frameworks. The student presentation estimation system in engineering institution is a motionless manual. Usually, the assessment of student’s performance using the traditional system is limited to the use of students’ performance scores, while failing to evaluate their performance based on activities or practical applications. In addition, such systems do not take cognizance of individual knowledge of students that connects to different activities within the learning environment. Recently, engineering institutions have started paying attention to evaluation solutions that are based on wireless networks and Internet of Things (IoT). Therefore, in this study, an automated system has been proposed for the assessment of engineering students. The proposed system is designed based on IoT and wireless communication networks with the aim of improving the process of virtual education. The data used in this study has been collected through the use of different IoT sensors within the premises of the college, and pre-processed using normalization. After the data was pre-processed, it was stored in cloud. In order to enable the classification of student’s activity, an Adaptive Layered Bayesian Belief Network (AL-BBN) classifier is proposed in this work. The student’s scores have been calculated using fuzzy logic, while Multi-Gradient Boosting Decision Tree (MGBDT) was proposed for decision making. The use of python simulation tool is employed in the implementation of the proposed system, and the evaluation of the performance benchmarks was done as well. Based on the findings of the study, the proposed conceptual model outperformed the existing ones in terms of improving the process of online learning.

Automated Artificial Intelligence Empowered White Blood Cells Classification燤odel

White blood cells (WBC) or leukocytes are a vital component of the blood which forms the immune system, which is accountable to fight foreign elements. The WBC images can be exposed to different data analysis approaches which categorize different kinds of WBC. Conventionally, laboratory tests are carried out to determine the kind of WBC which is erroneous and time consuming. Recently, deep learning (DL) models can be employed for automated investigation of WBC images in short duration. Therefore, this paper introduces an Aquila Optimizer with Transfer Learning based Automated White Blood Cells Classification (AOTL-WBCC) technique. The presented AOTL-WBCC model executes data normalization and data augmentation process (rotation and zooming) at the initial stage. In addition, the residual network (ResNet) approach was used for feature extraction in which the initial hyperparameter values of the ResNet model are tuned by the use of AO algorithm. Finally, Bayesian neural network (BNN) classification technique has been implied for the identification of WBC images into distinct classes. The experimental validation of the AOTL-WBCC methodology is performed with the help of Kaggle dataset. The experimental results found that the AOTL-WBCC model has outperformed other techniques which are based on image processing and manual feature engineering approaches under different dimensions.

Excavation of Molecular Subtypes of Endometrial Cancer Based on DNA Methylation.

Tumor heterogeneity makes the diagnosis and treatment of endometrial cancer difficult. As an important modulator of gene expression, DNA methylation can affect tumor heterogeneity and, therefore, provide effective information for clinical treatment. In this study, we explored specific prognostic clusters based on 482 examples of endometrial cancer methylation data in the TCGA database. By analyzing 4870 CpG clusters, we distinguished three clusters with different prognostics. Differences in DNA methylation levels are associated with differences in age, grade, clinical pathological staging, and prognosis. Subsequently, we screened out 264 specific hypermethylation and hypomethylation sites and constructed a prognostic model for Bayesian network classification, which corresponded to the classification of the test set to the classification results of the train set. Since the tumor microenvironment plays a key role in determining immunotherapy responses, we conducted relevant analyses based on clusters separated from DNA methylation data to determine the immune function of each cluster. We also predicted their sensitivity to chemotherapy drugs. Specific classifications of DNA methylation may help to address the heterogeneity of previously existing molecular clusters of endometrial cancer, as well as to develop more effective, individualized treatments.

Cost Overrun Risk Assessment and Prediction in Construction Projects: A Bayesian Network Classifier Approach

Cost overrun risks are declared to be dynamic and interdependent. Ignoring the relationship between cost overrun risks during the risk assessment process is one of the primary reasons construction projects go over budget. Conversely, recent studies have failed to account for potential interrelationships between risk factors in their machine learning (ML) models. Additionally, the presented ML models are not interpretable. Thus, this study contributes to the entire ML process using a Bayesian network (BN) classifier model by considering the possible interactions between predictors, which are cost overrun risks, to predict cost overrun and assess cost overrun risks. Furthermore, this study compared the BN classifier model’s performance accuracy to that of the Naive Bayes (NB) and decision tree (DT) models to determine the effect of considering possible correlations between cost overrun risks on prediction accuracy. Moreover, the most critical risks and their relationships are identified by interpreting the learned BN model. The results indicated that the 18 BN models demonstrated an average prediction accuracy of 78.86%, significantly higher than the NB and DT. The present study identified the most significant risks as an increase in the cost of materials, lack of knowledge and experience among human resources, and inflation.

Feature Selection Techniques for Big Data Analytics

Big data applications have tremendously increased due to technological developments. However, processing such a large amount of data is challenging for machine learning algorithms and computing resources. This study aims to analyze a large amount of data with classical machine learning. The influence of different random sampling techniques on the model performance is investigated by combining the feature selection techniques and machine learning classifiers. The experiments used two feature selection techniques: random subset and random projection. Two machine learning classifiers were also used: Naïve Bayes and Bayesian Network. This study aims to maximize the model performance by reducing the data dimensionality. In the experiments, 400 runs were performed by reducing the data dimensionality of a video dataset that was more than 40 GB. The results show that the overall performance fluctuates between 70% accuracy to 74% for using sampled and non-sample (all the data), a slight difference in performance compared to the non-sampled dataset. With the overall view of the results, the best performance among all combinations of experiments is recorded for combination 3, where the random subset technique and the Bayesian network classifier were used. Except for the round where 10% of the dataset was used, combination 1 has the best performance among all combinations.

Identification of informational and probabilistic independence by adaptive thresholding

The independence assumptions help Bayesian network classifier (BNC), e.g., Naive Bayes (NB), reduce structure complexity and perform surprisingly well in many real-world applications. Semi-naive Bayesian techniques seek to improve the classification performance by relaxing the attribute independence assumption. However, the study of dependence rather than independence has received more attention during the past decade and the validity of independence assumptions needs to be further explored. In this paper, a novel learning technique, called Adaptive Independence Thresholding (AIT), is proposed to automatically identify the informational independence and probabilistic independence. AIT can respectively tune the network topologies of BNC learned from training data and testing instance under the framework of target learning. Zero-one loss, bias, variance and conditional log likelihood are introduced to compare the classification performance in the experimental study. The extensive experimental results on a collection of 36 benchmark datasets from the UCI machine learning repository show that AIT is more effective than other learning techniques (such as structure extension, attribute weighting) and helps make the final BNCs achieve remarkable classification improvements.

From undirected dependence to directed causality: A novel Bayesian learning approach

Bayesian network (BN) is one of the most powerful probabilistic models in the field of uncertain knowledge representation and reasoning. During the past decade, numerous approaches have been proposed to build directed acyclic graph (DAG) as the structural specification of BN. However, for most Bayesian network classifiers (BNCs) the directed edges in DAG substantially represent assertions of conditional independence rather than causal relationships although the learned joint probability distributions may fit data well, thus they cannot be applied to causal reasoning. In this paper, conditional entropy is introduced to measure causal uncertainty due to its asymmetry characteristic, and heuristic search strategy is applied to build Bayesian causal tree (BCT) by identifying significant causalities. The resulting highly scalable topology can represent causal relationship in terms of causal science, and corresponding joint probability can fit training data in terms of data science. Then ensemble learning strategy is applied to build Bayesian causal forest (BCF) with a set of BCTs, each taking different attribute as the root node to represent root cause for causality analysis. Extensive experiments performed on 32 public datasets from the UCI machine learning repository show that BCF achieves outstanding classification performance compared to state-of-the-art single-model BNCs (e.g., CFWNB), ensemble BNCs (e.g., WATAN, IWAODE, WAODE-MI and TAODE) and non-Bayesian learners (e.g., SVM, k-NN, LR).

Learning causal Bayesian networks based on causality analysis for classification

Revealing causal information by analyzing purely observational data, known as causal discovery, has drawn much attention. To prove that the causal knowledge mined from data can be applied to facilitate various machine learning tasks (e.g., classification), we propose to measure, describe and evaluate the causalities in the framework of Bayesian network (BN) learning. In this paper, heuristic search strategy is applied to explore the causal interpretation in the form of directed acyclic graph (DAG) for classification. While adding directed edges to the DAG, we first introduce the log-likelihood equivalence assertion to make the learned joint probability encoded in BN approximates the true one, then introduce the causal dependence assertion to assess the rationality of the learned causal relationship. We perform a range of experiments on 35 datasets and empirically show that this novel algorithm demonstrates competitive classification performance and excellent causal interpretation compared to state-of-the-art Bayesian network classifiers (e.g. SKDB, WATAN, SLB, and TAODE).

The dark energy survey 5-yr photometrically identified type Ia supernovae

ABSTRACT As part of the cosmology analysis using Type Ia Supernovae (SN Ia) in the Dark Energy Survey (DES), we present photometrically identified SN Ia samples using multiband light curves and host galaxy redshifts. For this analysis, we use the photometric classification framework SuperNNovatrained on realistic DES-like simulations. For reliable classification, we process the DES SN programme (DES-SN) data and introduce improvements to the classifier architecture, obtaining classification accuracies of more than 98 per cent on simulations. This is the first SN classification to make use of ensemble methods, resulting in more robust samples. Using photometry, host galaxy redshifts, and a classification probability requirement, we identify 1863 SNe Ia from which we select 1484 cosmology-grade SNe Ia spanning the redshift range of 0.07 < z < 1.14. We find good agreement between the light-curve properties of the photometrically selected sample and simulations. Additionally, we create similar SN Ia samples using two types of Bayesian Neural Network classifiers that provide uncertainties on the classification probabilities. We test the feasibility of using these uncertainties as indicators for out-of-distribution candidates and model confidence. Finally, we discuss the implications of photometric samples and classification methods for future surveys such as Vera C. Rubin Observatory Legacy Survey of Space and Time.

Human and organizational failures analysis in process industries using FBN-HFACS model: Learning from a toxic gas leakage accident

This study used a hybrid technique of Fuzzy sets theory (FST), Bayesian network (BN), and Human Factors Analysis and Classification System (HFACS) to investigate a toxic gas leakage accident quantitatively. HFACS is one of the most popular techniques for assessing the contribution of human and organizational factors in industrial accidents. However, the technique is qualitative and unable to prioritize contributing factors. FST and BN are robust and flexible tools to quantify HFACS. The contributing factors were extracted from the interviews with all key persons and later were classified based on the HFACS taxonomy. The BN model was constructed based on HFACS structure and identified contributing factors. FST was used for estimating the effect rate of contributing factors on the accident. The conversion scale six was employed to gather experts’ opinions, the similarity aggregation method was used for aggregating these opinions, and defuzzification was conducted using the center of gravity method. The improvement index was calculated using the dynamic nature of the BN model for all contributing factors. According to the results, conflicts among several units within the organization, poor safety culture, using substandard equipment, lack of proper inspection on newly-purchased equipment, and several safety violations were the most critical factors contributing to the accident. The improvement index is a function of effect rate, structure of the BN, and organizational level within which the failure occurred, so it is superior to the effect rate for prioritizing contributing factors.

Development of an integrated blended learning model and its performance prediction on students’ learning using Bayesian network

With the spread of the COVID-19 pandemic, the importance of online learning has grown up worldwide and many higher education institutions used this mode of learning to save the timings of students. Just Online learning does not fulfill all the learning requirements of undergraduate students, therefore, there is a need for the blended learning (BL) method to be adopted in higher educational institutes for the enhancement of students’ learning outcomes. This research paper focuses on the development of an integrated blended learning model and the performance of the model on students’ learning has been predicted using a Bayesian network (BN) classifier. The proposed model is based on the medium impact blend of the Rotation model and the Enriched Virtual Model and applied to undergraduate computing students. The Data Structures and Algorithms course is targeted for the prediction of students’ performance. The findings of the proposed Integrated BL model show that when students properly attend the classroom lectures followed by their associated lab practical in the Rotation Model and follow the online learning activities in the Enriched Virtual Model properly, then their learning outcomes may be increased as predicted using BN method. The proposed model also reports an overall accuracy of 88.5% on the collected data.

Mining the Prognostic Role of DNA Methylation Heterogeneity in Lung Adenocarcinoma.

Purpose DNA methylation heterogeneity is a type of tumor heterogeneity in the tumor microenvironment, but studies on the identification of the molecular heterogeneity of the lung adenocarcinoma genome with respect to DNA methylation sites and their roles in lung cancer progression and prognosis are scarce. Methods Prognosis-associated DNA methylation subtypes were filtered by the Cox proportional hazards model and then established by unsupervised cluster analysis. Association analysis of these subtypes with clinical features and functional analysis of annotated genes potentially affected by methylation sites were performed. The robustness of the model was further tested by a Bayesian network classifier. Results Over 7 thousand methylation sites were associated with lung adenocarcinoma prognosis. We identified seven molecular methylation subtypes, including 630 methylation sites. The subtypes yielded the most stable results for differentiating methylation profiles, prognosis, and gene expression patterns. The annotated genes potentially affected by these methylation sites are enriched in biological processes such as morphogenesis and cell adhesion, but their individual impact on the tumor microenvironment and prognosis is multifaceted. Discussion. We revealed that DNA methylation heterogeneity could be clustered and associated with the clinical features and prognosis of lung adenocarcinoma, which could lead to the development of a novel molecular tool for clinical evaluation.

A machine-learning algorithm for distinguishing malignant from benign indeterminate thyroid nodules using ultrasound radiomic features.

Background: Ultrasound (US)-guided fine needle aspiration (FNA) cytology is the gold standard for the evaluation of thyroid nodules. However, up to 30% of FNA results are indeterminate, requiring further testing. In this study, we present a machine-learning analysis of indeterminate thyroid nodules on ultrasound with the aim to improve cancer diagnosis. Methods: Ultrasound images were collected from two institutions and labeled according to their FNA (F) and surgical pathology (S) diagnoses [malignant (M), benign (B), and indeterminate (I)]. Subgroup breakdown (FS) included: 90 BB, 83 IB, 70 MM, and 59 IM thyroid nodules. Margins of thyroid nodules were manually annotated, and computerized radiomic texture analysis was conducted within tumor contours. Initial investigation was conducted using five-fold cross-validation paradigm with a two-class Bayesian artificial neural networks classifier, including stepwise feature selection. Testing was conducted on an independent set and compared with a commercial molecular testing platform. Performance was evaluated using receiver operating characteristic analysis in the task of distinguishing between malignant and benign nodules. Results: About 1052 ultrasound images from 302 thyroid nodules were used for radiomic feature extraction and analysis. On the training/validation set comprising 263 nodules, five-fold cross-validation yielded area under curves (AUCs) of 0.75 [Standard Error (SE) = 0.04; ] and 0.67 (SE = 0.05; ) for the classification tasks of MM versus BB, and IM versus IB, respectively. On an independent test set of 19 IM/IB cases, the algorithm for distinguishing indeterminate nodules yielded an AUC value of 0.88 (SE = 0.09; ), which was higher than the AUC of a commercially available molecular testing platform (AUC = 0.81, SE = 0.11; ). Conclusion: Machine learning of computer-extracted texture features on gray-scale ultrasound images showed promising results classifying indeterminate thyroid nodules according to their surgical pathology.

Alleviating the attribute conditional independence and I.I.D. assumptions of averaged one-dependence estimator by double weighting

Learning Bayesian network classifiers (BNCs) from data is NP-hard. Of numerous BNCs, averaged one-dependence estimator (AODE) performs extremely well against more sophisticated newcomers, and its trade-off between bias and variance can be attributed to the independence assumption and i.i.d. assumption, which respectively address the issues of structure complexity and data complexity. To alleviate these assumptions and improve AODE, we propose to apply double weighting, including attribute weighting and model weighting, to finely tune the estimates of conditional probability based on generative learning and joint probability based on discriminative learning, respectively. Instance weighting is introduced to define the information-theoretic metrics for identifying the variation in probability distributions for different data points. This highly scalable learning approach can establish a decision boundary that is specifically tailored to each instance. Our extensive experimental evaluation on 34 datasets from the UCI machine learning repository shows that, attribute weighting and model weighting are complementary although they can work separately. The proposed AODE applying double weighting schema, called DWAODE, is a competitive alternative to other weighting approaches. The experimental results show that DWAODE demonstrates significant advantage in terms of zero–one loss, bias–variance decomposition, RMSE (root mean squared error), Friedman and Nemenyi tests.

An Interactive Online App for Predicting Diabetes via Machine Learning from Environment-Polluting Chemical Exposure Data.

The early prediction and identification of risk factors for diabetes may prevent or delay diabetes progression. In this study, we developed an interactive online application that provides the predictive probabilities of prediabetes and diabetes in 4 years based on a Bayesian network (BN) classifier, which is an interpretable machine learning technique. The BN was trained using a dataset from the Ansung cohort of the Korean Genome and Epidemiological Study (KoGES) in 2008, with a follow-up in 2012. The dataset contained not only traditional risk factors (current diabetes status, sex, age, etc.) for future diabetes, but it also contained serum biomarkers, which quantified the individual level of exposure to environment-polluting chemicals (EPC). Based on accuracy and the area under the curve (AUC), a tree-augmented BN with 11 variables derived from feature selection was used as our prediction model. The online application that implemented our BN prediction system provided a tool that performs customized diabetes prediction and allows users to simulate the effects of controlling risk factors for the future development of diabetes. The prediction results of our method demonstrated that the EPC biomarkers had interactive effects on diabetes progression and that the use of the EPC biomarkers contributed to a substantial improvement in prediction performance.

Selective AnDE based on attributes ranking by Maximin Conditional Mutual Information (MMCMI)

ABSTRACT Attribute selection has been proved to be an effective trick to strengthen the classification capability of Bayesian network classifiers, such as Averaged -Dependence Estimators (AnDE). However, conventional mutual information-based attribute ranking considers only the correlation between the attribute and the class, regardless of the redundancies among the attributes. In this paper, we propose a new ranking approach, called Maximin Conditional Mutual Information (MMCMI), which first minimises the conditional mutual information for any unsorted attribute with regard to the sorted attribute sequence, and then maximise the minimal conditional mutual information within all unsorted attributes. When ranking the very first attribute, the mutual information with the class is maximised within all attributes. Extensive empirical results demonstrate that the MMCMI ranking approach together with attribute selection framework achieves significantly superior classification performance and less classification time with respect to regular AnDE and the mutual information counterparts.

Explanation and Use of Uncertainty Quantified by Bayesian Neural Network Classifiers for Breast Histopathology Images.

Despite the promise of Convolutional neural network (CNN) based classification models for histopathological images, it is infeasible to quantify its uncertainties. Moreover, CNNs may suffer from overfitting when the data is biased. We show that Bayesian-CNN can overcome these limitations by regularizing automatically and by quantifying the uncertainty. We have developed a novel technique to utilize the uncertainties provided by the Bayesian-CNN that significantly improves the performance on a large fraction of the test data (about 6% improvement in accuracy on 77% of test data). Further, we provide a novel explanation for the uncertainty by projecting the data into a low dimensional space through a nonlinear dimensionality reduction technique. This dimensionality reduction enables interpretation of the test data through visualization and reveals the structure of the data in a low dimensional feature space. We show that the Bayesian-CNN can perform much better than the state-of-the-art transfer learning CNN (TL-CNN) by reducing the false negative and false positive by 11% and 7.7% respectively for the present data set. It achieves this performance with only 1.86 million parameters as compared to 134.33 million for TL-CNN. Besides, we modify the Bayesian-CNN by introducing a stochastic adaptive activation function. The modified Bayesian-CNN performs slightly better than Bayesian-CNN on all performance metrics and significantly reduces the number of false negatives and false positives (3% reduction for both). We also show that these results are statistically significant by performing McNemar's statistical significance test. This work shows the advantages of Bayesian-CNN against the state-of-the-art, explains and utilizes the uncertainties for histopathological images. It should find applications in various medical image classifications.

Stochastic optimization for bayesian network classifiers

Stochastic optimization for bayesian network classifiers

Measuring vulnerability to multidimensional poverty with Bayesian network classifiers

Bayesian network methods have recently gained great popularity in machine learning literature and applications to model uncertainty in complex phenomena that include relationships between multiple random variables. However, these models are not commonly applied in economics and development studies. Here, we introduce the Bayesian network classifier models to estimate the probability of a person to be welfare deprived in one and multiple dimensions. These probabilities are then used for measuring vulnerability to multidimensional poverty (VMP) in four alternative measurement frameworks. Currently, two of them can be found in the literature, but have been estimated with Probit and Logit models, which are unidimensional strategies. Instead of that, in this study, we follow a multidimensional strategy to solve an estimation problem that is multidimensional in nature. Two new VMP measurement procedures based on Bayesian network classifiers estimates are also introduced in this article. We illustrate the four estimation procedures using the household survey and the census data from Chile 2017. A 5-fold cross-validation exercise verifies a high predictive performance of these Bayesian network classifier models, with the highest accuracy being that of one of the new measurements that we put forward. Our findings reveal that the Bayesian network classifier models offer an adequate alternative to face the policy challenge of measuring vulnerability to multidimensional poverty.