Multi-agent distributed data mining approach for classifying meteorology data: case study on Iran’s synoptic weather stations

Abstract

Today’s organizations, industries and research centers are geographically distributed in different sites. To achieve true knowledge of business, mining massive amounts of data is necessary. In earth-related sciences such as meteorology, the date obtained from the various types of sensors is huge because of the high-frequency rate of data acquisition process and also the geographical distribution of weather stations. Therefore, the data mining and knowledge discovery process of this big and distributed data is a challenging work. In this paper, we propose a new distributed data mining approach called multi-agent hierarchical data mining to classify meteorology data, which has been collected from different sites widely distributed around the country (Iran). Our method utilizes a modified version of REPTree algorithm, which has been optimized to work in multi-agent system. To evaluate the proposed approach, it is implemented on 20 million of meteorology data record. Experimental results show that multi-agent hierarchical data mining approach can achieve significant performance improvement over centralized and parallel methods for knowledge discovery in large amounts of data.