Super-Parent-One-Dependence Estimators Research Articles

Instance-based weighting filter for superparent one-dependence estimators

Bayesian network classifiers remain of great interest in recent years, among which semi-naive Bayesian classifiers which utilize superparent one-dependence estimators (SPODEs) have shown superior predictive power. Linear weighting schemes are effective and efficient ones for linearly combining SPODEs, whereas it is a challenging task for averaged one-dependence estimators (AODE) to find globally optimal and fixed weights for its SPODE members. The joint probability distribution of SPODE may not always fit different test instances to the same extent, thus a flexible rather than rigid weighting scheme would be a feasible solution for the final AODE to approximate the true joint probability distribution. Based on this promise, we propose a novel instance-based weighting filter, which can flexibly assign discriminative weights to each single SPODE for different test instances. Meanwhile, the weight considers not only the mutual dependence between the superparent and class variable, but also the conditional dependence between the superparent and non-superparent attributes. Experimental comparison on 30 publicly available datasets shows that SPODE with instance-based weighting filter outperforms state-of-the-art BNCs with and without weighting methods in terms of zero–one loss, bias and variance with minimal additional computation.

SODE: Self-Adaptive One-Dependence Estimators for classification

SuperParent-One-Dependence Estimators (SPODEs) represent a family of semi-naive Bayesian classifiers which relax the attribute independence assumption of Naive Bayes (NB) to allow each attribute to depend on a common single attribute (superparent). SPODEs can effectively handle data with attribute dependency but still inherent NB׳s key advantages such as computational efficiency and robustness for high dimensional data. In reality, determining an optimal superparent for SPODEs is difficult. One common approach is to use weighted combinations of multiple SPODEs, each having a different superparent with a properly assigned weight value (i.e., a weight value is assigned to each attribute). In this paper, we propose a self-adaptive SPODEs, namely SODE, which uses immunity theory in artificial immune systems to automatically and self-adaptively select the weight for each single SPODE. SODE does not need to know the importance of individual SPODE nor the relevance among SPODEs, and can flexibly and efficiently search optimal weight values for each SPODE during the learning process. Extensive experiments and comparisons on 56 benchmark data sets, and validations on image and text classification, demonstrate that SODE outperforms state-of-the-art weighted SPODE algorithms and is suitable for a wide range of learning tasks. Results also confirm that SODE provides an appropriate balance between runtime efficiency and accuracy.

Efficient learning ensemble SuperParent-one-dependence estimator by maximizing conditional log likelihood

The ensemble of SuperParent one-dependence estimators (SPODEs) is one of the most effective improved algorithms. It achieves high classification accuracy while decreasing variance. However, most existing approaches only focus on performance improvement of individual SPODEs in selection and weighting procedures but overlook the importance of the entire ensemble model.Based on the assumption that the performance of the entire ensemble classifier can obtain better weight distribution than using the greedy strategy inside each SPODE, we propose an ensemble SPODE algorithm by maximizing conditional log likelihood (EODE-CLL). First, we choose the maximum conditional probability as the global optimization goal, which can avoid over-fitting problem compared with the least squares error. Second, the algorithm assigns hierarchical weights for SPODEs and the attributes inside SPODE. The second weight layer can help fully optimize local SPODE model. Finally, stochastic gradient descent method is used to search best parameters. It has good scalability, which has spawned batch and distributed version. Compared to the existing ensemble SPODEs, our proposed model achieves more accurate and robust classification results, while shows better time complexity.We conduct experiments on a public benchmark containing 36 datasets. The results of the experiments show that our EODE-CLL significantly outperforms state-of-the-art ensemble SPODE methods in terms of accuracy, F-measure, bias, and variance.

One-Dependence Estimators for Accurate Detection of Anomalous Network Traffic

Classification of Internet traffic as anomalous has re- mained an issue of concern for the network security re- search community in recent times. Advances in comput- ing performance, in terms of processing power and stor- age, have allowed the use of resource-intensive intelligent algorithms, to detect intrusive activities, in a timely manner. Na¨ ive Bayes is a statistical inference learning algorithm with promise for document classification, spam detection and intrusion detection. The attribute independence issue associated with NaBayes has been resolved through the development of one dependence estimation algorithms such as Super-Parent One Dependence Estimators (SPODE) and Average One Dependence Estimator (AODE). In this pa- per, we propose the design of an intrusion detection sys- tem based on the classification capabilities of SPODE and AODE for accurate detection of anomalous network traffic. The performance of the proposed scheme is studied and an- alyzed on the KDD-99 intrusion benchmark data set, with significantly high detection rates of the two algorithms as opposed to results obtained through Na¨ ive Bayes simula- tion.

Credal ensembles of classifiers

It is studied how to aggregate the probabilistic predictions generated by different SPODE (Super-Parent-One-Dependence Estimators) classifiers. It is shown that aggregating such predictions via compression-based weights achieves a slight but consistent improvement of performance over previously existing aggregation methods, including Bayesian Model Averaging and simple average (the approach adopted by the AODE algorithm). Then, attention is given to the problem of choosing the prior probability distribution over the models; this is an important issue in any Bayesian ensemble of models. To robustly deal with the choice of the prior, the single prior over the models is substituted by a set of priors over the models (credal set), thus obtaining a credal ensemble of Bayesian classifiers. The credal ensemble recognizes the prior-dependent instances, namely the instances whose most probable class varies when different prior over the models are considered. When faced with prior-dependent instances, the credal ensemble remains reliable by returning a set of classes rather than a single class. Two credal ensembles of SPODEs are developed; the first generalizes the Bayesian Model Averaging and the second the compression-based aggregation. Extensive experiments show that the novel ensembles compare favorably to traditional methods for aggregating SPODEs and also to previous credal classifiers.

To Select or To Weigh: A Comparative Study of Linear Combination Schemes for SuperParent-One-Dependence Estimators

We conduct a large-scale comparative study on linearly combining superparent-one-dependence estimators (SPODEs), a popular family of seminaive Bayesian classifiers. Altogether, 16 model selection and weighing schemes, 58 benchmark data sets, and various statistical tests are employed. This paper's main contributions are threefold. First, it formally presents each scheme's definition, rationale, and time complexity and hence can serve as a comprehensive reference for researchers interested in ensemble learning. Second, it offers bias-variance analysis for each scheme's classification error performance. Third, it identifies effective schemes that meet various needs in practice. This leads to accurate and fast classification algorithms which have an immediate and significant impact on real-world applications. Another important feature of our study is using a variety of statistical tests to evaluate multiple learning methods across multiple data sets.