Abstract
The naive Bayesian classification method has received significant attention in the field of supervised learning. This method has an unrealistic assumption in that it views all attributes as equally important. Attribute weighting is one of the methods used to alleviate this assumption and consequently improve the performance of the naive Bayes classification. This study, with a focus on nonlinear optimization problems, proposes four attribute weighting methods by minimizing four different loss functions. The proposed loss functions belong to a family of exponential functions that makes the optimization problems more straightforward to solve, provides analytical properties of the trained classifier, and allows for the simple modification of the loss function such that the naive Bayes classifier becomes robust to noisy instances. This research begins with a typical exponential loss which is sensitive to noise and provides a series of its modifications to make naive Bayes classifiers more robust to noisy instances. Based on numerical experiments conducted using 28 datasets from the UCI machine learning repository, we confirmed that the proposed scheme successfully determines optimal attribute weights and improves the classification performance.
Highlights
Based on the Bayesian decision theorem, a Bayesian classifier predicts a test instance as a class that has the highest membership probability
The weighted naive Bayes based on the gradient-based L-BFGSM method [24] was proposed by introducing two objective functions: conditional log-likelihood (CLL) and mean square error (MSE) [25], which focus on maximizing the likelihood of data from a probability perspective and minimizing predictive error from a classification perspective
The average RI for the proposed methods was computed by taking the largest RI value from each row in the last four columns (ENB, deviance naive Bayes (DNB), likelihood naive Bayes (LNB), and generalized DNB (GDNB)) and averaging them because this study provides four options to perform attribute weighting
Summary
Based on the Bayesian decision theorem, a Bayesian classifier predicts a test instance as a class that has the highest membership probability. To obtain weights that are optimal with respect to classification performance, wrapper approaches have been proposed They assign attribute weights or select a subset of attributes in a heuristic optimization manner by utilizing a learning algorithm of interest in the training step with the learning objective of maximizing predictive power. The weighted naive Bayes based on the gradient-based L-BFGSM method [24] was proposed by introducing two objective functions: conditional log-likelihood (CLL) and mean square error (MSE) [25], which focus on maximizing the likelihood of data from a probability perspective and minimizing predictive error from a classification perspective This method was extended by giving different attribute weights to different class labels [26].
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
