Distributed feature extraction in a p2p setting — a case study

Abstract

Finding the right data representation is essential for virtually every data mining application. In this work we describe an approach to collaborative feature extraction, selection and aggregation in distributed, loosely coupled domains. In contrast to other work in the field of distributed data mining, we focus on scenarios in which a large number of loosely coupled nodes apply data mining to different, usually very small and overlapping, subsets of the entire data space. The aim is not to find a global concept to cover all data, but to learn a set of local concepts. Our prototypical application is a distributed media organization platform, called Nemoz, that assists users in maintaining their media collections. We propose two models for collaborative feature extraction, selection and aggregation for supervised data mining. One is based on a centralized p2p architecture, and the other on a fully distributed p2p architecture. We compare both models on a real world data set and discuss their advantages and problems.