Hidden Naive Bayes Research Articles

A Novel Mixed-Attribute Fusion-Based Naive Bayesian Classifier

The Naive Bayesian classifier (NBC) is a well-known classification model that has a simple structure, low training complexity, excellent scalability, and good classification performances. However, the NBC has two key limitations: (1) it is built upon the strong assumption that condition attributes are independent, which often does not hold in real-life, and (2) the NBC does not handle continuous attributes well. To overcome these limitations, this paper presents a novel approach for NBC construction, called mixed-attribute fusion-based NBC (MAF-NBC). It alleviates the two aforementioned limitations by relying on a mixed-attribute fusion mechanism with an improved autoencoder neural network for NBC construction. MAF-NBC transforms the original mixed attributes of a data set into a series of encoded attributes with maximum independence as a pre-processing step. To guarantee the generation of useful encoded attributes, an efficient objective function is designed to optimize the weights of the autoencoder neural network by considering both the encoding error and the attribute’s dependence. A series of persuasive experiments was conducted to validate the feasibility, rationality, and effectiveness of the designed MAF-NBC approach. Results demonstrate that MAF-NBC has superior classification performance than eight state-of-the-art Bayesian algorithms, namely the discretization-based NBC (Dis-NBC), flexible naive Bayes (FNB), tree-augmented naive (TAN) Bayes, averaged one-dependent estimator (AODE), hidden naive Bayes (HNB), deep feature weighting for NBC (DFW-NBC), correlation-based feature weighting filter for NBC (CFW-NBC), and independent component analysis-based NBC (ICA-NBC).

A Novel Hybrid Approach: Instance Weighted Hidden Naive Bayes

Naive Bayes (NB) is easy to construct but surprisingly effective, and it is one of the top ten classification algorithms in data mining. The conditional independence assumption of NB ignores the dependency between attributes, so its probability estimates are often suboptimal. Hidden naive Bayes (HNB) adds a hidden parent to each attribute, which can reflect dependencies from all the other attributes. Compared with other Bayesian network algorithms, it offers significant improvements in classification performance and avoids structure learning. However, the assumption that HNB regards each instance equivalent in terms of probability estimation is not always true in real-world applications. In order to reflect different influences of different instances in HNB, the HNB model is modified into the improved HNB model. The novel hybrid approach called instance weighted hidden naive Bayes (IWHNB) is proposed in this paper. IWHNB combines instance weighting with the improved HNB model into one uniform framework. Instance weights are incorporated into the improved HNB model to calculate probability estimates in IWHNB. Extensive experimental results show that IWHNB obtains significant improvements in classification performance compared with NB, HNB and other state-of-the-art competitors. Meanwhile, IWHNB maintains the low time complexity that characterizes HNB.

Adapting Hidden Naive Bayes for Text Classification

Due to its simplicity, efficiency, and effectiveness, multinomial naive Bayes (MNB) has been widely used for text classification. As in naive Bayes (NB), its assumption of the conditional independence of features is often violated and, therefore, reduces its classification performance. Of the numerous approaches to alleviating its assumption of the conditional independence of features, structure extension has attracted less attention from researchers. To the best of our knowledge, only structure-extended MNB (SEMNB) has been proposed so far. SEMNB averages all weighted super-parent one-dependence multinomial estimators; therefore, it is an ensemble learning model. In this paper, we propose a single model called hidden MNB (HMNB) by adapting the well-known hidden NB (HNB). HMNB creates a hidden parent for each feature, which synthesizes all the other qualified features’ influences. For HMNB to learn, we propose a simple but effective learning algorithm without incurring a high-computational-complexity structure-learning process. Our improved idea can also be used to improve complement NB (CNB) and the one-versus-all-but-one model (OVA), and the resulting models are simply denoted as HCNB and HOVA, respectively. The extensive experiments on eleven benchmark text classification datasets validate the effectiveness of HMNB, HCNB, and HOVA.

Steel strip surface inspection through the combination of feature selection and multiclass classifiers

Purpose During the production process of steel strip, some defects may appear on the surface, that is, traditional manual inspection could not meet the requirements of low-cost and high-efficiency production. The purpose of this paper is to propose a method of feature selection based on filter methods combined with hidden Bayesian classifier for improving the efficiency of defect recognition and reduce the complexity of calculation. The method can select the optimal hybrid model for realizing the accurate classification of steel strip surface defects. Design/methodology/approach A large image feature set was initially obtained based on the discrete wavelet transform feature extraction method. Three feature selection methods (including correlation-based feature selection, consistency subset evaluator [CSE] and information gain) were then used to optimize the feature space. Parameters for the feature selection methods were based on the classification accuracy results of hidden Naive Bayes (HNB) algorithm. The selected feature subset was then applied to the traditional NB classifier and leading extended NB classifiers. Findings The experimental results demonstrated that the HNB model combined with feature selection approaches has better classification performance than other models of defect recognition. Among the results of this study, the proposed hybrid model of CSE + HNB is the most robust and effective and of highest classification accuracy in identifying the optimal subset of the surface defect database. Originality/value The main contribution of this paper is the development of a hybrid model combining feature selection and multi-class classification algorithms for steel strip surface inspection. The proposed hybrid model is primarily robust and effective for steel strip surface inspection.

Bayes Performance of Batch Data Mining Based on Functional Dependencies

Batch processes and phenomena in traffic video data processing, such as traffic video image processing and intelligent transportation, are commonly used. The application of batch processing can increase the efficiency of resource conservation. However, owing to limited research on traffic video data processing conditions, batch processing activities in this area remain minimally examined. By employing database functional dependency mining, we developed in this study a workflow system. Meanwhile, the Bayesian network is a focus area of data mining. It provides an intuitive means for users to comply with causality expression approaches. Moreover, graph theory is also used in data mining area. In this study, the proposed approach depends on relational database functions to remove redundant attributes, reduce interference, and select a property order. The restoration of selective hidden naive Bayesian (SHNB) affects this property order when it is used only once. With consideration of the hidden naive Bayes (HNB) influence, rather than using one pair of HNB, it is introduced twice. We additionally designed and implemented mining dependencies from a batch traffic video processing log for data execution algorithms.

Enhancing the performance of association rule models by filtering instances in colorectal cancer patients

Colorectal cancer data available from the SEER program is analyzed with the aim of using filtering techniques to improve the performance of association rule models. In this paper, it is proposed to improve the quality of the dataset by removing its outliers using the Hidden Naive Bayes (HNB), Naive Bayes Tree (NBTree ) and Reduced Error Pruning Decision Tree (REPTree) algorithms. The Apriori and HotSpot algorithms are applied to mine the association rules between the 13 selected attributes and average survivals. Experimental results show that the HNB algorithm can improve the accuracy of the Apriori algorithm’s performance by up to 100% and support threshold up to 45%. It can also improve the accuracy of the HotSpot algorithm’s performance up to 93.38% and support threshold up to 80%. Therefore, the HotSpot rules with minimum support of 80% are selected for explanation. The HotSpot algorithm shows that colorectal cancer patients, who died from colon cancer and were not receiving radiation therapy, were associated with survival of less than 22 months. Our study shows that filtering techniques in the preprocessing stage are a useful approach in enhancing the quality of the data set. This finding could help researchers build models for better prediction and performance analysis. Although it is heuristic, such analysis can be very useful to identify the factors affecting survival. It can also aid medical practitioners in helping patients to understand risks involved in a particular treatment procedure.

Modified Mahalanobis Taguchi System for Imbalance Data Classification.

The Mahalanobis Taguchi System (MTS) is considered one of the most promising binary classification algorithms to handle imbalance data. Unfortunately, MTS lacks a method for determining an efficient threshold for the binary classification. In this paper, a nonlinear optimization model is formulated based on minimizing the distance between MTS Receiver Operating Characteristics (ROC) curve and the theoretical optimal point named Modified Mahalanobis Taguchi System (MMTS). To validate the MMTS classification efficacy, it has been benchmarked with Support Vector Machines (SVMs), Naive Bayes (NB), Probabilistic Mahalanobis Taguchi Systems (PTM), Synthetic Minority Oversampling Technique (SMOTE), Adaptive Conformal Transformation (ACT), Kernel Boundary Alignment (KBA), Hidden Naive Bayes (HNB), and other improved Naive Bayes algorithms. MMTS outperforms the benchmarked algorithms especially when the imbalance ratio is greater than 400. A real life case study on manufacturing sector is used to demonstrate the applicability of the proposed model and to compare its performance with Mahalanobis Genetic Algorithm (MGA).

Hidden Naive Bayes Indoor Fingerprinting Localization Based on Best-Discriminating AP Selection

Indoor fingerprinting localization approaches estimate the location of a mobile object by matching observations of received signal strengths (RSS) from access points (APs) with fingerprint records. In real WLAN environments, there are more and more APs available, with interference between them, which increases the localization difficulty and computational consumption. To cope with this, a novel AP selection method, LocalReliefF-C( a novel method based on ReliefF and correlation coefficient), is proposed. Firstly, on each reference location, the positioning capability of APs is ranked by calculating classification weights. Then, redundant APs are removed via computing the correlations between APs. Finally, the set of best-discriminating APs of each reference location is obtained, which will be used as the input features when the location is estimated. Furthermore, an effective clustering method is adopted to group locations into clusters according to the common subsets of the best-discriminating APs of these locations. In the online stage, firstly, the sequence of RSS observations is collected to calculate the set of the best-discriminating APs on the given location, which is subsequently used to compare with cluster keys in order to determine the target cluster. Then, hidden naive Bayes (HNB) is introduced to estimate the target location, which depicts the real WLAN environment more accurately and takes into account the mutual interaction of the APs. The experiments are conducted in the School of Environmental Science and Spatial Informatics at the China University of Mining and Technology. The results validate the effectiveness of the proposed methods on improving localization accuracy and reducing the computational consumption.

An Evolutionary Bayesian Network Learning Algorithm using Feature Subset Selection for Bayesian Network Classifiers

Classification is the process of constructing (learning) a model (classifier) to predict the class (labels) for given data. Bayesian belief network classifiers allow the representation of dependencies between subsets of attributes of a dataset. But learning an optimal Bayesian belief network for a Bayesian network classifier is a NPhard problem. A number of heuristic-based algorithms have been proposed for supervised learning of bayesian belief network such as Tree-Augmented Naive Bayes (TAN), Hidden Naive Bayes (HNB). In the past decade, swarm intelligence (SI) based algorithms have been proposed for many optimization problems. Swarm intelligence based algorithms are characterized by the collective decentralized decision-making of several independent agents to search for the optimal solution in the solution search space. In this paper, we propose a hybrid swarm intelligence based Bayesian network classifier combining Hunting Group search with Feature subset selections. The Hunting Group Search - Feature Subset Selection (HuGS-FSS) algorithm has been inspired by the behaviour of a pack of hunting animals such as wolves. The classification accuracy of the proposed HuGSFSS algorithm is tested against other state of the art Bayesian network classifiers such as Naive Bayes, TAN, A2De, and HNB. Through comprehensive evaluation using 28 benchmark classification datasets from the UCI repository, we show that HuGS-FSS outperforms the other state of the art algorithms.

An intrusion detection and alert correlation approach based on revising probabilistic classifiers using expert knowledge

Bayesian networks are important knowledge representation tools for handling uncertain pieces of information. The success of these models is strongly related to their capacity to represent and handle dependence relations. Some forms of Bayesian networks have been successfully applied in many classification tasks. In particular, naive Bayes classifiers have been used for intrusion detection and alerts correlation. This paper analyses the advantage of adding expert knowledge to probabilistic classifiers in the context of intrusion detection and alerts correlation. As examples of probabilistic classifiers, we will consider the well-known Naive Bayes, Tree Augmented Naive Bayes (TAN), Hidden Naive Bayes (HNB) and decision tree classifiers. Our approach can be applied for any classifier where the outcome is a probability distribution over a set of classes (or decisions). In particular, we study how additional expert knowledge such as "it is expected that 80 % of traffic will be normal" can be integrated in classification tasks. Our aim is to revise probabilistic classifiers' outputs in order to fit expert knowledge. Experimental results show that our approach improves existing results on different benchmarks from intrusion detection and alert correlation areas.

A Comparison Study of Bayesian Classifiers on Web Pages Classification

With the development of internet, web mining has become a hotspot of data mining. The first step of web mining is to classify web pages into interesting classes, so the classification is one of the essential techniques for web mining. In this paper, we study the capabilities of bayesian classifiers for web pages categorization, after that we report our work on the comparison of binary-classification and multi-classification. Results on benchmark dataset show that bayesian classifiers perform satisfying, especially for Hidden Naive Bayes (HNB) which is more competitive than other methods. Furthermore, the performances of binary-classification are better than those of multi-classification under the evaluation metrics of accuracy and F-measure.

A Novel Bayes Model: Hidden Naive Bayes

Because learning an optimal Bayesian network classifier is an NP-hard problem, learning-improved naive Bayes has attracted much attention from researchers. In this paper, we summarize the existing improved algorithms and propose a novel Bayes model: hidden naive Bayes (HNB). In HNB, a hidden parent is created for each attribute which combines the influences from all other attributes. We experimentally test HNB in terms of classification accuracy, using the 36 UCI data sets selected by Weka, and compare it to naive Bayes (NB), selective Bayesian classifiers (SBC), naive Bayes tree (NBTree), tree-augmented naive Bayes (TAN), and averaged one-dependence estimators (AODE). The experimental results show that HNB significantly outperforms NB, SBC, NBTree, TAN, and AODE. In many data mining applications, an accurate class probability estimation and ranking are also desirable. We study the class probability estimation and ranking performance, measured by conditional log likelihood (CLL) and the area under the ROC curve (AUC), respectively, of naive Bayes and its improved models, such as SBC, NBTree, TAN, and AODE, and then compare HNB to them in terms of CLL and AUC. Our experiments show that HNB also significantly outperforms all of them.