Multi-agent approach for data mining-based bagging ensembles to improve the decision process for big data

Abstract

Today, data growth is accelerating to create a big data in various fields, such as social media, websites, e-mails, finance, and medicine. It needs analysis and knowledge extraction. In addition, data mining is a technology whose purpose is to promote information and knowledge extraction from a big data. In this paper, a multi-layered approach based on agents is proposed to extract knowledge from big dataset with bagging algorithm. To achieve this, we call the paradigm of a multi-agent system in Hadoop to distribute the complexity and processing of large datasets across several autonomous entities called agents. The goal is to predict the target class or value for each case in the data using the bagging technique that is dedicated to the task of classification or regression. This proposition will help decision-makers to take right decisions and provide a perfect response time by the use of the multi-agent system in Hadoop. Therefore, to implement the proposed architecture, it is more convenient to use the Apache Hadoop framework, Apache Spark MLlib framework for building scalable machine learning algorithms and JADE platform which provides a complete set of services and agents.