Learning from examples—a uniform view

Abstract

We compare different empirical learning methods regarding their strategy of reduction of the subset with important examples in a measurement space. In this way a uniform view of AI and statistical methods alike is presented. Theoretically they are all based on the Bayes' classifier. They all construct a classification rule which partitions measurement space into target sets. Advantages and drawbacks of different methods are highlightened by simple examples. Error analyses enable deeper understanding of the learning and classification process in real world domains, characterized by incomplete and noisy data. Good error estimate is based on the balance between bias and variance. A deviation of the Laplacean error estimate is presented. We presented new mechanisms that use redundant knowledge in the explicit form. One of such systems, GINESYS (Generic INductive Expert SYstem Shell) is shortly presented. Heuristic reasoning and empirical results indicate that a proper use of redundant knowledge significantly increases classification accuracy. Over 10 basic AI and statistical systems were tested on two oncological domains. Results show that older AI methods provide usable information regarding the structure of data, but, on the other hand, their classification accuracy is often lower than that of the statistical methods. Standard statistical systems often achieve good classification accuracy, but are more or less non-transparent to users. Some new AI systems construct robust redundant knowledge, provide explanations in a humanly understandable way and outperform the classification accuracy of standard statistical methods.