Naive Bayes Learning Research Articles

International Conference on Computational Science, ICCS 2011 Farmer-Pest Problem: A Multidimensional Problem Domain for Comparison of Agent Learning Methods

Abstract Learning is often utilized by multi-agent systems which can deal with complex problems by means of their decentralized approach. With a number of learning methods available, a need for their comparison arises. This paper presents initial comparison results for selected algorithms (SARSA, Naive Bayes, C4.5 and Ripper), which are obtained based on the new multi-dimensional Farmer-Pest problem domain, which is suitable for benchmarking learning algorithms. The results show that supervised learning algorithms can be used to generate agent strategy. It appears that for simple environment reinforcement learning algorithm together with Naive Bayes learning gives best results. Although, in a difficult environment, C4.5 and Ripper are the best.

Mining a corpus of biographical texts using keywords

Using statistically derived keywords to characterize texts has become an important research method for digital humanists and corpus linguists in areas such as literary analysis and the exploration of genre difference. Keywords—and the associated concepts of 'keyness' and `key-keyness'-have inspired conferences and workshops, many and varied research papers, and are central to several modern corpus processing tools. In this article, we present evidence that (at least for the task of biographical sentence classification) frequent words characterize texts better than keywords or key-keywords. Using the naive Bayes learning algorithm in conjunction with frequency-, keyword-, and key-keyword-based text representation to classify a corpus of biographical sentences, we discovered that the use of frequent words alone provided a classification accuracy better than either the keyword or key-keyword representations at a statistically significant level. This result suggests that (for the biographical sentence classification task at least) frequent words characterize texts better than keywords derived using more computationally intensive methods.

점진적 특징 가중치 기법을 이용한 나이브 베이즈 문서분류기의 성능 개선

In the real-world operational environment, most of text classification systems have the problems of insufficient training documents and no prior knowledge of feature space. In this regard, Naive Bayes is known to be an appropriate algorithm of operational text classification since the classification model can be evolved easily by incrementally updating its pre-learned classification model and feature space. This paper proposes the improving technique of Naive Bayes classifier through feature weighting strategy. The basic idea is that parameter estimation of Naive Bayes considers the degree of feature importance as well as feature distribution. We can develop a more accurate classification model by incorporating feature weights into Naive Bayes learning algorithm, not performing a learning process with a reduced feature set. In addition, we have extended a conventional feature update algorithm for incremental feature weighting in a dynamic operational environment. To evaluate the proposed method, we perform the experiments using the various document collections, and show that the traditional Naive Bayes classifier can be significantly improved by the proposed technique.

Discretization for naive-Bayes learning: managing discretization bias and variance

Quantitative attributes are usually discretized in Naive-Bayes learning. We establish simple conditions under which discretization is equivalent to use of the true probability density function during naive-Bayes learning. The use of different discretization techniques can be expected to affect the classification bias and variance of generated naive-Bayes classifiers, effects we name discretization bias and variance. We argue that by properly managing discretization bias and variance, we can effectively reduce naive-Bayes classification error. In particular, we supply insights into managing discretization bias and variance by adjusting the number of intervals and the number of training instances contained in each interval. We accordingly propose proportional discretization and fixed frequency discretization, two efficient unsupervised discretization methods that are able to effectively manage discretization bias and variance. We evaluate our new techniques against four key discretization methods for naive-Bayes classifiers. The experimental results support our theoretical analyses by showing that with statistically significant frequency, naive-Bayes classifiers trained on data discretized by our new methods are able to achieve lower classification error than those trained on data discretized by current established discretization methods.

Integrating Naïve Bayes and FOIL

A novel relational learning approach that tightly integrates the naive Bayes learning scheme with the inductive logic programming rule-learner FOIL is presented. In contrast to previous combinations that have employed naive Bayes only for post-processing the rule sets, the presented approach employs the naive Bayes criterion to guide its search directly. The proposed technique is implemented in the NFOIL and TFOIL systems, which employ standard naive Bayes and tree augmented naive Bayes models respectively. We show that these integrated approaches to probabilistic model and rule learning outp erform post-processing approaches. They also yield significantly more accurate models than si mple rule learning and are competitive with more sophisticated ILP systems.

An evaluation of Naive Bayes variants in content-based learning for spam filtering

We describe an in-depth analysis of spam-filtering performance of a simple Naive Bayes learner and two extended variants. A set of seven mailboxes comprising about 65,000 mails from seven different users, as well as a representative snapshot of 25,000 mails which were received over 18 weeks by a single user, were used for evaluation. Our main motivation was to test whether two extended variants of Naive Bayes learning, SA-Train and CRM114, were superior to simple Naive Bayes learning, represented by SpamBayes. Surprisingly, we found that the performance of these systems was remarkably similar and that the extended systems have significant weaknesses which are not apparent for the simpler Naive Bayes learner. The simpler Naive Bayes learner, SpamBayes, also offers the most stable performance in that it deteriorates least over time. Overall, SpamBayes should be preferred over the more complex variants.

선호도 재계산을 위한 연관 사용자 군집 분석과 Representative Attribute -Neighborhood를 이용한 협력적 필터링 시스템의 성능향상

추천 시스템에 있어서 협력적 필터링 기술은 많은 연구가 되고 있다. 그러나 협력적 필터링 기술을 이용한 추천 시스템은 초기 평가 문제와 희박성 문제가 발생한다. 이를 해결하기 위해서 본 논문에서는 선호도 재 계산을 위한 연관 사용자 군집과 베이지안 추정치를 이용한 사용자 선호도 예측 방법을 제안한다. 제안한 방법에서는 협력적 필터링 시스템에서 아이템의 속성을 고려하지 않는 단점을 보완하기 위해서 선호도에 가장 크게 영향을 미치는 대표 장르를 추출하여 유사한 이웃을 찾아 낼 때 예측에 이용하는 Representative Attribute-Neighborhood 방법을 사용한다. 협력적 필터링의 알고리즘에 군집 아이템 백터 내의 특정 아이템의 선호도를 재계산 하기 위한 연관 사용자 군집 분석을 적용하여 성능 향상을 하였다. 또 초기 평가 문제와 희박성 문제를 해결하기 위하여 Association Rule Hypergraph Partitioning 알고리즘을 사용하여 사용자를 장르별로 군집한다. 새로운 사용자는 Naive Bayes 분류자에 의해 이들 장르 중 하나로 분류된다. 또한, 분류된 장르 내에 속한 사용자들과 새로운 사용자의 유사도를 구하기 위해 Naive Bayes 학습을 통해 사용자가 평가한 아이템에 추정치를 달리 부여한다. 추정치가 부여된 선호도를 피어슨 상관 관계에 적용할 경우 결측치(Missing Value)로 인한 예측의 오류를 적게하여 예측의 정확도를 높일 수 있다. 제안된 방법은 기존의 방법보다 높은 성능을 나타냄을 보인다. There has been much research focused on collaborative filtering technique in Recommender System. However, these studies have shown the First-Rater Problem and the Sparsity Problem. The main purpose of this Paper is to solve these Problems. In this Paper, we suggest the user's predicting preference method using Bayesian estimated value and the associative user clustering for the recalculation of preference. In addition to this method, to complement a shortcoming, which doesn't regard the attribution of item, we use Representative Attribute-Neighborhood method that is used for the prediction when we find the similar neighborhood through extracting the representative attribution, which most affect the preference. We improved the efficiency by using the associative user's clustering analysis in order to calculate the preference of specific item within the cluster item vector to the collaborative filtering algorithm. Besides, for the problem of the Sparsity and First-Rater, through using Association Rule Hypergraph Partitioning algorithm associative users are clustered according to the genre. New users are classified into one of these genres by Naive Bayes classifier. In addition, in order to get the similarity value between users belonged to the classified genre and new users, and this paper allows the different estimated value to item which user evaluated through Naive Bayes learning. As applying the preference granted the estimated value to Pearson correlation coefficient, it can make the higher accuracy because the errors that cause the missing value come less. We evaluate our method on a large collaborative filtering database of user rating and it significantly outperforms previous proposed method.

Conditionally independent component analysis for supervised feature extraction

The present paper extends the framework of independent component analysis (ICA) to supervised learning. The key idea is to find a conditionally independent representation of input variables for a target variable by linear transformation. The representation can be considered as independent components of observations from which explanatory parts for a target variable are removed, and it is directly useful for naive Bayes learning which has been reported to perform as well as more sophisticated methods for prediction. The learning algorithm is derived under a similar but different criterion to ICA. The algorithm attempts to maximize the independence among extracted features as well as the mutual information between extracted features and a target variable.

Technical Note: Naive Bayes for Regression

Despite its simplicity, the naive Bayes learning scheme performs well on most classification tasks, and is often significantly more accurate than more sophisticated methods. Although the probability estimates that it produces can be inaccurate, it often assigns maximum probability to the correct class. This suggests that its good performance might be restricted to situations where the output is categorical. It is therefore interesting to see how it performs in domains where the predicted value is numeric, because in this case, predictions are more sensitive to inaccurate probability estimates.This paper shows how to apply the naive Bayes methodology to numeric prediction (i.e., regression) tasks by modeling the probability distribution of the target value with kernel density estimators, and compares it to linear regression, locally weighted linear regression, and a method that produces “model trees”—decision trees with linear regression functions at the leaves. Although we exhibit an artificial dataset for which naive Bayes is the method of choice, on real-world datasets it is almost uniformly worse than locally weighted linear regression and model trees. The comparison with linear regression depends on the error measure: for one measure naive Bayes performs similarly, while for another it is worse. We also show that standard naive Bayes applied to regression problems by discretizing the target value performs similarly badly. We then present empirical evidence that isolates naive Bayes' independence assumption as the culprit for its poor performance in the regression setting. These results indicate that the simplistic statistical assumption that naive Bayes makes is indeed more restrictive for regression than for classification.