Averaged One-Dependence Estimators Research Articles

Flexible model weighting for one-dependence estimators based on point-wise independence analysis

Recent studies have shown that Bayesian network classifiers (BNCs) are powerful tools for knowledge representation and classification, and averaged one-dependence estimators (AODE) is one of the most popular and effective BNCs since it can achieve the tradeoff between bias and variance due to its independence assumptions and ensemble learning strategy. However, unverified independence assumptions may result in biased estimates of probability distribution and then degradation in classification performance. In this paper, we prove theoretically the uncertainty of probability-theoretic independence and propose to measure the independence between attribute values implicated in specific instance. The estimates of conditional probability can be finely tuned based on point-wise independence analysis. Point-wise log likelihood function is then applied as weighting metric for committee members of AODE to improve the estimate of joint probability. Extensive experiments on 36 benchmark datasets show that, compared to other state-of-the-art classifiers, weighted one-dependence estimators using point-wise independence analysis can achieve competitive classification performance in terms of zero-one loss, RMSE, bias-variance decomposition and conditional log likelihood.

Interaction of Secure Cloud Network and Crowd Computing for Smart City Data Obfuscation.

There can be many inherent issues in the process of managing cloud infrastructure and the platform of the cloud. The platform of the cloud manages cloud software and legality issues in making contracts. The platform also handles the process of managing cloud software services and legal contract-based segmentation. In this paper, we tackle these issues directly with some feasible solutions. For these constraints, the Averaged One-Dependence Estimators (AODE) classifier and the SELECT Applicable Only to Parallel Server (SELECT-APSL ASA) method are proposed to separate the data related to the place. ASA is made up of the AODE and SELECT Applicable Only to Parallel Server. The AODE classifier is used to separate the data from smart city data based on the hybrid data obfuscation technique. The data from the hybrid data obfuscation technique manages 50% of the raw data, and 50% of hospital data is masked using the proposed transmission. The analysis of energy consumption before the cryptosystem shows the total packet delivered by about 71.66% compared with existing algorithms. The analysis of energy consumption after cryptosystem assumption shows 47.34% consumption, compared to existing state-of-the-art algorithms. The average energy consumption before data obfuscation decreased by 2.47%, and the average energy consumption after data obfuscation was reduced by 9.90%. The analysis of the makespan time before data obfuscation decreased by 33.71%. Compared to existing state-of-the-art algorithms, the study of makespan time after data obfuscation decreased by 1.3%. These impressive results show the strength of our methodology.

Alleviating the attribute conditional independence and I.I.D. assumptions of averaged one-dependence estimator by double weighting

Learning Bayesian network classifiers (BNCs) from data is NP-hard. Of numerous BNCs, averaged one-dependence estimator (AODE) performs extremely well against more sophisticated newcomers, and its trade-off between bias and variance can be attributed to the independence assumption and i.i.d. assumption, which respectively address the issues of structure complexity and data complexity. To alleviate these assumptions and improve AODE, we propose to apply double weighting, including attribute weighting and model weighting, to finely tune the estimates of conditional probability based on generative learning and joint probability based on discriminative learning, respectively. Instance weighting is introduced to define the information-theoretic metrics for identifying the variation in probability distributions for different data points. This highly scalable learning approach can establish a decision boundary that is specifically tailored to each instance. Our extensive experimental evaluation on 34 datasets from the UCI machine learning repository shows that, attribute weighting and model weighting are complementary although they can work separately. The proposed AODE applying double weighting schema, called DWAODE, is a competitive alternative to other weighting approaches. The experimental results show that DWAODE demonstrates significant advantage in terms of zero–one loss, bias–variance decomposition, RMSE (root mean squared error), Friedman and Nemenyi tests.

A method of predicting oil and gas resource spatial distribution based on Bayesian network and its application

Abstract The spatial distribution prediction of hydrocarbon resource is essential to reduce exploration risks and improve investment returns. Determining the location of the spatial distribution of oil and gas resources is a complex and uncertain problem. This paper systematically analyzes oil and gas exploration risk assessment and resource spatial distribution prediction technology, and proposes a method of predicting hydrocarbon spatial distribution based on averaged one-dependence estimators(AODE). Firstly, the transformation process of petroleum geological problem to mathematical model is described. Then, the classification principle and decision rule of AODE model are expounded. Finally, a case study of Sangonghe Formation in the hinterland of Junggar Basin in China is given to further illustrate the proposed method and work flow. The case prediction results not only reveal that the accuracy of the AODE model can reach 85.2% on the data set of 203 exploratory wells, which is higher than state-of-the-art methods (such as tree augmented Bayesian network method and the Mahalanobis distance method), but also point out the areas with high probability of remaining hydrocarbon resources in Sangonghe Formation, which provides decision-making basis for the next exploration. The application results show that the AODE model can effectively predict the spatial distribution of oil and gas and visualize the risk of geological exploration, thereby optimizing drilling strategy and increasing economic benefits.

Alleviating the independence assumptions of averaged one-dependence estimators by model weighting

Of numerous proposals to refine naive Bayes by weakening its attribute independence assumption, averaged one-dependence estimators (AODE) has been shown to be able to achieve significantly higher classification accuracy at a moderate cost in classification efficiency. However, all one-dependence estimators (ODEs) in AODE have the same weights and are treated equally. To address this issue, model weighting, which assigns discriminate weights to ODEs and then linearly combine their probability estimates, has been proved to be an efficient and effective approach. Most information-theoretic weighting metrics, including mutual information, Kullback-Leibler measure and the information gain, place more emphasis on the correlation between root attribute (value) and class variable. We argue that the topology of each ODE can be divided into a set of local directed acyclic graphs (DAGs) based on the independence assumption, and multivariate mutual information is introduced to measure the extent to which the DAGs fit data. Based on this premise, in this study we propose a novel weighted AODE algorithm, called AWODE, that adaptively selects weights to alleviate the independence assumption and make the learned probability distribution fit the instance. The proposed approach is validated on 40 benchmark datasets from UCI machine learning repository. The experimental results reveal that, AWODE achieves bias-variance trade-off and is a competitive alternative to single-model Bayesian learners (such as TAN and KDB) and other weighted AODEs (such as WAODE).

Semi-supervised weighting for averaged one-dependence estimators

Semi-supervised weighting for averaged one-dependence estimators

Averaged tree-augmented one-dependence estimators

Ever since the success of naive Bayes (NB) in achieving excellent classification performance and the least computational overhead, more and more researchers have focused their attention on the Bayesian network classifiers (BNCs). Among numerous approaches to refining NB, averaged one-dependence estimators (AODE) achieves excellent classification performance although its discriminative independence assumption for each member rarely holds in practice. Robust AODE with high expressivity and low bias is in urgent need with the ever increasing data quantity. In this paper, the log likelihood function $LL({\mathscr{B}}|D)$ is introduced to measure the number of bits which is encoded in the network topology ${\mathscr{B}}$ for describing training data D. An efficient heuristic search strategy is applied to maximize $LL({\mathscr{B}}|D)$ and relax the independence assumption of AODE by exploring higher-order conditional dependencies between attributes. The proposed approach, averaged tree-augmented one-dependence estimators (ATODE), inherits the effectiveness of AODE and gains more flexibility for modelling higher-order dependencies. The extensive experimental comparison results on 36 datasets demonstrate that, compared to state-of-the-art learners including single-model BNCs (e.g., CFWNB and SKDB) and variants of AODE (e.g., TAODE), our proposed out-of-core learner can achieve competitive or better classification performance.

Instance-based weighting filter for superparent one-dependence estimators

Bayesian network classifiers remain of great interest in recent years, among which semi-naive Bayesian classifiers which utilize superparent one-dependence estimators (SPODEs) have shown superior predictive power. Linear weighting schemes are effective and efficient ones for linearly combining SPODEs, whereas it is a challenging task for averaged one-dependence estimators (AODE) to find globally optimal and fixed weights for its SPODE members. The joint probability distribution of SPODE may not always fit different test instances to the same extent, thus a flexible rather than rigid weighting scheme would be a feasible solution for the final AODE to approximate the true joint probability distribution. Based on this promise, we propose a novel instance-based weighting filter, which can flexibly assign discriminative weights to each single SPODE for different test instances. Meanwhile, the weight considers not only the mutual dependence between the superparent and class variable, but also the conditional dependence between the superparent and non-superparent attributes. Experimental comparison on 30 publicly available datasets shows that SPODE with instance-based weighting filter outperforms state-of-the-art BNCs with and without weighting methods in terms of zero–one loss, bias and variance with minimal additional computation.

Computational Learning Approaches for Personalized Pregnancy Care

The increasing use of interconnected sensors to monitor patients with chronic diseases, integrated with tools for the management of shared information, can guarantee a better performance of health information systems (HISs) by performing personalized healthcare. The early diagnosis of chronic diseases such as hypertensive disorders of pregnancy represents a significant challenge in women’s healthcare. Computational learning techniques are useful tools for pattern recognition in the assessment of an increasing amount of integrated data related to these diseases. Hence, in this paper, the use of machine learning (ML) techniques is proposed for the assessment of real data referred to hypertensive disorders in pregnancy. The results show that the averaged one-dependence estimator algorithm can help in the decision- making process in uncertain moments, thus improving the early detection of these chronic diseases. The best-evaluated computational learning algorithm improves the performance of HISs through its precise diagnosis. This method can be applied in electronic health (e-health) environments as a useful tool for handling uncertainty in the decision-making process related to high-risk pregnancy.

Self-Adaptive Attribute Value Weighting for Averaged One-Dependence Estimators

Of numerous proposals for weakening the attribute independence assumption of Naive Bayes, averaged one-dependence estimators (AODE) learns by extrapolation from marginal to full-multivariate probability distributions, and has demonstrated reasonable improvement in terms of classification performance. However, all the one-dependence estimators in AODE are assigned with the same weight, and their probability estimates are combined linearly. This work presents an efficient and effective attribute value weighting approach that assigns discriminative weights to different super-parent one-dependence estimators for different instances by identifying the differences among these one-dependence estimators in terms of log likelihood. The proposed approach is validated on widely used benchmark datasets from UCI machine learning repository. Experimental results show that the proposed approach achieves bias-variance trade-off and is a competitive alternative to state-of-the-art Bayesian and non-Bayesian learners (e.g., tree augmented Naive Bayes and logistic regression).

Synthetic Minority Oversampling Technique pada Averaged One Dependence Estimators untuk Klasifikasi Credit Scoring

Bank or financial institution is a business entity whose activities are collecting funds from the public in the form of deposits and channeling them to the public in the form of credit and or other forms. In credit financing problems often occur and one of the problems faced in credit assessment is imbalance class data sets or dataset class imbalances. This problem can be overcome by resampling method, namely by using Oversampling, undersampling and hybrids that combine the two sampling approaches. This research proposes the method of applying SMOTE or Synthetic Minority Oversampling Technique on Averaged One Dependence estimators (AODE) to improve the performance of the accuracy of the credit rating classification on German Credit Creditetsets. The results of this experimental study on the GermanCredit dataset with the classification method without the Resampling process on 13 classifiers produce an average performance value of 70%. The results of the classification with classification techniques that apply the SMOTE method on the AODE algorithm can increase the accuracy performance by 5.5% with an accuracy value of 0.817 or 81.69%. While the classification technique that applies the SpreadSubSample + AODE method decreased by 0.041 or 4.1% but still higher than the accuracy value of other methods with an accuracy value of 0.723 or 72.33%. The researcher concludes that by applying the Resampling technique with the SMOTE method on the AODE algorithm can increase the value of accuracy performance effectively on the imbalance class used for credit scoring or credit rating on GermanCredit datasets.

Biomedical data analytics in mobile-health environments for high-risk pregnancy outcome prediction

According to the World Health Organization (WHO), a significant reduction in mortality and maternal morbidity has occurred in developed countries over the past decades. In contrast, these rates remain high in developing countries. Smart mobile-health (m-health) applications that use machine learning (ML) approaches are necessary tools for pregnancy monitoring in an accessible, reliable, and cost-efficient manner, making the prediction of high-risk situations possible during gestation. This paper, therefore, proposes the development, performance evaluation, and comparison of ML algorithms based on Bayesian networks capable of identifying at-risk pregnancies based on the symptoms and risk factors presented by the patients. A performance comparison of several Bayes-based ML algorithms determined the best-suited algorithm for the prediction, identification, and accompaniment of hypertensive disorders during pregnancy. The contribution of this study focuses on finding a smart classifier for the development of novel mobile devices, which presents reliable results in the identification of problems related to pregnancy. Through the well-known cross-validation method, this proposal is evaluated and compared with other recent approaches. The averaged one-dependence estimators presented better results on average than the other approaches. These findings are key to improving the health monitoring of women suffering from high-risk pregnancies around the world. Thus, this study can contribute to a reduction of both maternal and fetal deaths.

Averaged one-dependence estimators on edge devices for smart pregnancy data analysis

The development of edge computing has allowed the Internet of Things (IoT) to reach higher levels of operational efficiency, knowledge generation, and decision-making through a better relationship between applications and their users. The principal idea is the processing of large amounts of data close to the network edge, where these data are generated. Hence, in this paper, the use of a machine learning technique, known as averaged one-dependence estimators, is proposed for real-time pregnancy data analysis from IoT devices and gateways. This statistical technique is useful for decentralized pre-processing of data and its intermediate storage, reducing the amount of data to be transferred to the cloud and ensuring operability, even in an event of network failure. The results show that this technique presents accurate results with a low computational cost and could be a useful tool to better take advantage of the potential of IoT solutions in healthcare.

Combining multi-objective evolutionary algorithm with averaged one-dependence estimators for big data analytics

Even though many researchers tried to explore the various possibilities on multi-objective feature selection, still, it is yet to be explored with best of its capabilities in data mining applications rather than going for developing new ones. In this paper, multi-objective evolutionary algorithm ENORA is used to select the features in a multi-class classification problem. The fusion of averaged n-dependence estimators (AnDE) with n = 1, a variant of naive Bayes with efficient feature selection by ENORA is performed in order to obtain a fast hybrid classifier which can effectively learn from big data. This method aims at solving the problem of finding optimal feature subset from full data which at present still remains to be a difficult problem. The efficacy of the obtained classifier is extensively evaluated with a range of most popular 21 real world dataset, ranging from small to big ones. The results obtained are encouraging in terms of time; root means square error; zero-one loss and classification accuracy.

Combining multi-objective evolutionary algorithm with averaged one-dependence estimators for big data analytics

Even though many researchers tried to explore the various possibilities on multi-objective feature selection, still, it is yet to be explored with best of its capabilities in data mining applications rather than going for developing new ones. In this paper, multi-objective evolutionary algorithm ENORA is used to select the features in a multi-class classification problem. The fusion of averaged n-dependence estimators (AnDE) with n = 1, a variant of naive Bayes with efficient feature selection by ENORA is performed in order to obtain a fast hybrid classifier which can effectively learn from big data. This method aims at solving the problem of finding optimal feature subset from full data which at present still remains to be a difficult problem. The efficacy of the obtained classifier is extensively evaluated with a range of most popular 21 real world dataset, ranging from small to big ones. The results obtained are encouraging in terms of time; root means square error; zero-one loss and classification accuracy.

K-Dependence Bayesian Classifier Ensemble

To maximize the benefit that can be derived from the information implicit in big data, ensemble methods generate multiple models with sufficient diversity through randomization or perturbation. A k-dependence Bayesian classifier (KDB) is a highly scalable learning algorithm with excellent time and space complexity, along with high expressivity. This paper introduces a new ensemble approach of KDBs, a k-dependence forest (KDF), which induces a specific attribute order and conditional dependencies between attributes for each subclassifier. We demonstrate that these subclassifiers are diverse and complementary. Our extensive experimental evaluation on 40 datasets reveals that this ensemble method achieves better classification performance than state-of-the-art out-of-core ensemble learners such as the AODE (averaged one-dependence estimator) and averaged tree-augmented naive Bayes (ATAN).

Attribute Value Weighted Average of One-Dependence Estimators

Of numerous proposals to improve the accuracy of naive Bayes by weakening its attribute independence assumption, semi-naive Bayesian classifiers which utilize one-dependence estimators (ODEs) have been shown to be able to approximate the ground-truth attribute dependencies; meanwhile, the probability estimation in ODEs is effective, thus leading to excellent performance. In previous studies, ODEs were exploited directly in a simple way. For example, averaged one-dependence estimators (AODE) weaken the attribute independence assumption by directly averaging all of a constrained class of classifiers. However, all one-dependence estimators in AODE have the same weights and are treated equally. In this study, we propose a new paradigm based on a simple, efficient, and effective attribute value weighting approach, called attribute value weighted average of one-dependence estimators (AVWAODE). AVWAODE assigns discriminative weights to different ODEs by computing the correlation between the different root attribute value and the class. Our approach uses two different attribute value weighting measures: the Kullback–Leibler (KL) measure and the information gain (IG) measure, and thus two different versions are created, which are simply denoted by AVWAODE-KL and AVWAODE-IG, respectively. We experimentally tested them using a collection of 36 University of California at Irvine (UCI) datasets and found that they both achieved better performance than some other state-of-the-art Bayesian classifiers used for comparison.

Attribute weighting for averaged one-dependence estimators

Averaged one-dependence estimators (AODE) is a type of supervised learning algorithm that relaxes the conditional independence assumption that governs standard naive Bayes learning algorithms. AODE has demonstrated reasonable improvement in terms of classification performance when compared with a naive Bayes learner. However, AODE does not consider the relationships between the super-parent attribute and other normal attributes. In this paper, we propose a novel method based on AODE that weighs the relationship between the attributes called weighted AODE (WAODE), which is an attribute weighting method that uses the conditional mutual information metric to rank the relations among the attributes. We have conducted experiments on University of California, Irvine (UCI) benchmark datasets and compared accuracies between AODE and our proposed learner. The experimental results in our paper show that WAODE exhibits higher accuracy performance than the original AODE.

Bayes classifiers for imbalanced traffic accidents datasets

Traffic accidents data sets are usually imbalanced, where the number of instances classified under the killed or severe injuries class (minority) is much lower than those classified under the slight injuries class (majority). This, however, supposes a challenging problem for classification algorithms and may cause obtaining a model that well cover the slight injuries instances whereas the killed or severe injuries instances are misclassified frequently. Based on traffic accidents data collected on urban and suburban roads in Jordan for three years (2009–2011); three different data balancing techniques were used: under-sampling which removes some instances of the majority class, oversampling which creates new instances of the minority class and a mix technique that combines both. In addition, different Bayes classifiers were compared for the different imbalanced and balanced data sets: Averaged One-Dependence Estimators, Weightily Average One-Dependence Estimators, and Bayesian networks in order to identify factors that affect the severity of an accident. The results indicated that using the balanced data sets, especially those created using oversampling techniques, with Bayesian networks improved classifying a traffic accident according to its severity and reduced the misclassification of killed and severe injuries instances. On the other hand, the following variables were found to contribute to the occurrence of a killed causality or a severe injury in a traffic accident: number of vehicles involved, accident pattern, number of directions, accident type, lighting, surface condition, and speed limit. This work, to the knowledge of the authors, is the first that aims at analyzing historical data records for traffic accidents occurring in Jordan and the first to apply balancing techniques to analyze injury severity of traffic accidents.

Averaged Extended Tree Augmented Naive Classifier

This work presents a new general purpose classifier named Averaged Extended Tree Augmented Naive Bayes (AETAN), which is based on combining the advantageous characteristics of Extended Tree Augmented Naive Bayes (ETAN) and Averaged One-Dependence Estimator (AODE) classifiers. We describe the main properties of the approach and algorithms for learning it, along with an analysis of its computational time complexity. Empirical results with numerous data sets indicate that the new approach is superior to ETAN and AODE in terms of both zero-one classification accuracy and log loss. It also compares favourably against weighted AODE and hidden Naive Bayes. The learning phase of the new approach is slower than that of its competitors, while the time complexity for the testing phase is similar. Such characteristics suggest that the new classifier is ideal in scenarios where online learning is not required.