Ranking of Classifiers based on Dataset Characteristics using Active Meta Learning

Abstract

Classification is a machine learning technique which is used to categorize the different input patterns into different classes. To select the best classifier for a given dataset is one of the critical issues in Classification. Using cross-validation approach, it is possible to apply candidate algorithms on a given dataset and best classifier is selected by considering various evaluation measures of Classification. But computational cost is significant. Meta Learning automates this process by acquiring knowledge in form of Meta-features and performance information of candidate algorithm on datasets and creates a Meta Knowledge Base. Once Meta Knowledge Base is generated, system uses k-Nearest Neighbor as a Meta Learner that identifies the most similar datasets to new dataset. But generation of Meta Example is a costly process due to a large number of candidate algorithms and datasets with different characteristics involved. So Active Learning is incorporated into Meta Learning System that reduces generation of Meta example and at the same time maintaining performance of candidate algorithms. Once the training phase is completed based on Active Meta Learning approach, ranking is provided based on Success Rate Ratio (SRR) method that considers accuracy as a performance evaluation measure.