Dimensionality reduction in automatic knowledge acquisition: a simple greedy search approach

Abstract

Knowledge acquisition is the process of collecting domain knowledge, documenting the knowledge, and transforming it into a computerized representation. Due to the difficulties involved in eliciting knowledge from human experts, knowledge acquisition was identified as a bottleneck in the development of knowledge-based system. Over the past decades, a number of automatic knowledge acquisition techniques have been developed. However, the performance of these techniques suffers from the so called curse of dimensionality, i.e., difficulties arise when many irrelevant (or redundant) parameters exist. This paper presents a heuristic approach based on statistics and greedy search for dimensionality reduction to facilitate automatic knowledge acquisition. The approach deals with classification problems. Specifically, Chi-square statistics are used to rank the importance of individual parameters. Then, a backward search procedure is employed to eliminate parameters (less important parameters first) that do not contribute to class separability. The algorithm is very efficient and was found to be effective when applied to a variety of problems with different characteristics.