Agent based preprocessing

Abstract

The current data mining tools is used to build knowledge based on a huge historical data. At present, businesses are facing with fast growing data that are very valuable in contributing knowledge. Knowledge should be updated regularly in order to ensure its quality and precision thus improve the decision making process. Data mining has shown great potential in extracting valuable knowledge from large databases. However, current data mining algorithms and tools are costly and several are too complex in their operations when dealing with large databases. In recent years, agents have become a popular paradigm in computing, because its autonomous, flexible and provides intelligence. Embedding agents in the current data mining processes and tools are believed to be able to solve the obstacle. One of the most important process in data mining is data preprocessing. It is reported that 60% of the data mining project is on preprocessing. Data preprocessing involves integration, selection, cleaning and transformation of data set that will be used for mining. This paper focuses on an agent-based preprocessing framework. The aims is to provides an auto preprocessing a set of new data, which suite to data mining novice user. The proposed agent based preprocessing framework consists of seven agents: User Interface agents, Coordinator Agent, Identify Agent, CleanMiss Agent, CleanNoisy Agent, Transformation Agent and Discretization Agent. User Interface Agent is designed in such a way to provide interface suite to novice users. Coordinator agent is responsible for coordinating and cooperating with all other agents to achieve the goals. Identify agent responsible to provide an adaptive user data cleaning profiling. CleanMiss Agent, CleanNoisy Agent, Transformation Agent and Discretization Agent provide various types of techniques autonomously, which ended with proposing the best cleaning techniques from various types of techniques to keep in the preprocessing profile. This paper is start by introducing the data mining process problem includes data preprocessing which agent can solve data mining problems. By applying agent in data preprocessing, a tool that intelligence yet flexible can be produced.