Good Feature Selection Research Articles

FIVE NEW FEATURE SELECTION METRICS IN TEXT CATEGORIZATION

Feature selection has been extensively applied in statistical pattern recognition as a mechanism for cleaning up the set of features that are used to represent data and as a way of improving the performance of classifiers. Four schemes commonly used for feature selection are Exponential Searches, Stochastic Searches, Sequential Searches, and Best Individual Features. The most popular scheme used in text categorization is Best Individual Features as the extremely high dimensionality of text feature spaces render the other three feature selection schemes time prohibitive. This paper proposes five new metrics for selecting Best Individual Features for use in text categorization. Their effectiveness have been empirically tested on two well- known data collections, Reuters-21578 and 20 Newsgroups. Experimental results show that the performance of two of the five new metrics, Bayesian Rule and F-one Value, is not significantly below that of a good traditional text categorization selection metric, Document Frequency. The performance of another two of these five new metrics, Low Loss Dimensionality Reduction and Relative Frequency Difference, is equal to or better than that of conventional good feature selection metrics such as Mutual Information and Chi-square Statistic.

Fault diagnostics of roller bearing using kernel based neighborhood score multi-class support vector machine

Roller bearing is one of the most widely used rotary elements in a rotary machine. The roller bearing’s nature of vibration reveals its condition and the features that show the nature are to be extracted through some indirect means. Statistical parameters like kurtosis, standard deviation, maximum value, etc. form a set of features, which are widely used in fault diagnostics. Finding out good features that discriminate the different fault conditions of the bearing is often a problem. Selection of good features is an important phase in pattern recognition and requires detailed domain knowledge. This paper addresses the feature selection process using decision tree and uses kernel based neighborhood score multi-class support vector machine (MSVM) for classification. The vibration signal from a piezoelectric transducer is captured for the following conditions: good bearing, bearing with inner race fault, bearing with outer race fault, and inner and outer race faults. The statistical features are extracted therefrom and classified successfully using MSVM. The results of MSVM are compared with and binary support vector machine (SVM).

Feature selection using Decision Tree and classification through Proximal Support Vector Machine for fault diagnostics of roller bearing

Roller bearing is one of the most widely used rotary elements in a rotary machine. The roller bearing's nature of vibration reveals its condition and the features that show the nature, are to be extracted through some indirect means. Statistical parameters like kurtosis, standard deviation, maximum value, etc. form a set of features, which are widely used in fault diagnostics. Often the problem is, finding out good features that discriminate the different fault conditions of the bearing. Selection of good features is an important phase in pattern recognition and requires detailed domain knowledge. This paper illustrates the use of a Decision Tree that identifies the best features from a given set of samples for the purpose of classification. It uses Proximal Support Vector Machine (PSVM), which has the capability to efficiently classify the faults using statistical features. The vibration signal from a piezoelectric transducer is captured for the following conditions: good bearing, bearing with inner race fault, bearing with outer race fault, and inner and outer race fault. The statistical features are extracted therefrom and classified successfully using PSVM and SVM. The results of PSVM and SVM are compared.

Relevance feedback for CBIR: a new approach based on probabilistic feature weighting with positive and negative examples

In content-based image retrieval, understanding the user's needs is a challenging task that requires integrating him in the process of retrieval. Relevance feedback (RF) has proven to be an effective tool for taking the user's judgement into account. In this paper, we present a new RF framework based on a feature selection algorithm that nicely combines the advantages of a probabilistic formulation with those of using both the positive example (PE) and the negative example (NE). Through interaction with the user, our algorithm learns the importance he assigns to image features, and then applies the results obtained to define similarity measures that correspond better to his judgement. The use of the NE allows images undesired by the user to be discarded, thereby improving retrieval accuracy. As for the probabilistic formulation of the problem, it presents a multitude of advantages and opens the door to more modeling possibilities that achieve a good feature selection. It makes it possible to cluster the query data into classes, choose the probability law that best models each class, model missing data, and support queries with multiple PE and/or NE classes. The basic principle of our algorithm is to assign more importance to features with a high likelihood and those which distinguish well between PE classes and NE classes. The proposed algorithm was validated separately and in image retrieval context, and the experiments show that it performs a good feature selection and contributes to improving retrieval effectiveness.

An anomaly intrusion detection method by clustering normal user behavior

For detecting an intrusion based on the anomaly of a user's activities, previous works are concentrated on statistical techniques or frequent episode mining in order to analyze an audit data set. However, since they mainly analyze the average behavior of a user’s activities, some anomalies can be detected inaccurately. This paper proposes an anomaly detection method which utilizes a clustering algorithm for modeling the normal behavior of a user’s activities in a host. Since clustering can identify an arbitrary number of dense ranges in an analysis domain, it can eliminate the inaccuracy caused by statistical analysis. Consequently, it can model the frequent activities of a user more accurately than the statistical analysis does. The common knowledge of activities in the transactions of a user is represented by the occurrence frequency of similar activities by the unit of a transaction as well as the repetitive ratio of similar activities in each transaction. The proposed method also addresses how to maintain identified common knowledge as a concise profile. Furthermore, this paper addresses the selection of good features that can improve the detection rate of anomalous behavior in an on-line transaction.

Scalable feature mining for sequential data

Many real world data sets contain irrelevant or redundant attributes. This might be because the data was collected without data mining in mind or without a priori knowledge of the attribute dependences. Many data mining methods such as classification and clustering degrade prediction accuracy when trained on data sets containing redundant or irrelevant attributes or features. Selecting the right feature set not only can improve accuracy but also can reduce the running time of the predictive algorithms and lead to simpler, more understandable models. Good feature selection is thus a fundamental data preprocessing step in data mining. To provide good feature selection for sequential domains, we developed FeatureMine, a scalable feature mining algorithm that combines two powerful data mining paradigms: sequence mining and classification algorithms. Tests on three practical domains demonstrate that FeatureMine can efficiently handle very large data sets with thousands of items and millions of records.

Object Localization with Multiple Sensors

We present a method to obtain the position and orientation of an object through measurement from multiple sensors. Raw sensor measurements are subject to limitations of sensor precision and accuracy. Although for most measurements the estimate of position parameters is a linear function of the measurements, the estimate of orientation parameters is a nonlinear function of the measurements. Thus, error in orien tation estimate depends on the distance over which the raw measurements are made. For example, the estimate of the orientation of a line is better, the farther apart two points on the line are. The problem of finding the orientation parame ters is formulated in two steps. The first step computes vec tors from sensor measurements of points. A concept of best features is developed to select an optimal set of all possible vectors. The second step relates the orientation parameters to the vectors from the first step as a linear system. The best estimate is obtained by solving a weighted linear system of the optimal set of vectors in a least squares sense. The opti mal selection of best features improves the estimate substan tially. This method has been implemented for localizing an object in a manipulator end-effector instrumented with cen troid and matrix tactile sensors.

Tool Failure Monitoring in Turning by Pattern Recognition Analysis of AE Signals

Sensing of both gradual and catastrophic tool failure is a key aspect in producing high quality parts on fully automated machine tool systems. Acoustic emission provides a means of sensing tool failure, since it is generated from the processes that cause tool failure (e.g., tool wear, tool fracture). A linear discriminant function-based technique for detection of tool wear, tool fracture, or chip disturbance events is developed using the spectra of signals generated by these sources. In addition, a methodology for determining the feature dimensionality, the selection of best features, and the minimum training sample size is presented. The concepts of classification error minimization and manufacturing cost minimization have been applied to design classifiers using a hierarchical decision strategy to improve the performance of tool failure sensing. Results of an application indicate an 84 to 94 percent reliability for detecting tool failure of any type.