Presence Of Training Data Research Articles

Learning to Weight for Text Classification

In information retrieval (IR) and related tasks, term weighting approaches typically consider the frequency of the term in the document and in the collection in order to compute a score reflecting the importance of the term for the document. In tasks characterized by the presence of training data (such as text classification) it seems logical that the term weighting function should take into account the distribution (as estimated from training data) of the term across the classes of interest. Although “supervised term weighting” approaches that use this intuition have been described before, they have failed to show consistent improvements. In this article, we analyze the possible reasons for this failure, and call consolidated assumptions into question. Following this criticism, we propose a novel supervised term weighting approach that, instead of relying on any predefined formula, learns a term weighting function optimized on the training set of interest; we dub this approach Learning to Weight (LTW). The experiments that we run on several well-known benchmarks, and using different learning methods, show that our method outperforms previous term weighting approaches in text classification.

COMPARISON OF TRADITIONAL AND MACHINE LEARNING BASE METHODS FOR GROUND POINT CLOUD LABELING

Abstract. Today, a variety of methods have been proposed by researchers to distinguish ground and non-ground points in point cloud data. Most fully automated methods have a common disadvantage which is the lack of proper algorithm response for all areas and levels of the ground, so most of these algorithms have good outcomes in simple landscapes but encounter problems in complex landscapes. Point cloud filtering techniques can be divided into two general rule-based and novel methods. Today, the use of machine learning techniques has improved the results of classification, which has led to significant results, especially when data can be labelled at the presence of training data. In this paper, firstly, altimeter and radiometric features are extracted from the LiDAR data and the point cloud derived from digital photogrammetry. Then, these features are participated in a classification process using SVM learning and random forest methods, and the ground and Non-ground points are classified. The classification results using this method on LiDAR data show a total error of 6.2%, a type I error of 5.4%, and a type II error of 13.2%. The comparison of the proposed method with the results of LASTools software shows a reduction in total error and type I error (while increasing the type II error). This method was also investigated on the dense point cloud obtained from digital photogrammetry and based on this study, the total was 7.2%, the type I error was 6.8%, and the type II error was 10.9%.