Combination of Two Evolutionary Methods for Mining Association Rules in Large and Dense Databases

Abstract

Among several methods of extracting association rules that have been reported, a new evolutionary method named Genetic Network Programming (GNP) has also shown its effectiveness for small databases in the sense that they have a relatively small number of attributes. However, this conventional GNP method is not be able to deal with large databases with a huge number of attributes, because its search space becomes very large, causing bad performance at running time. The aim of this paper is to propose a new method to extract association rules from large and dense databases with a huge amount of attributes through the combination of conventional GNP based mining method and a specially designed genetic algorithm (GA). Each of these evolutionary methods works in its own processing level and they are highly synchronized to act as one system.Our strategy consists in the division of a large and dense database into many small databases. These small databases are considered as individuals and form a population. Then the conventional GNP based mining method is applied to extract association rules for each of these individuals. Finally, the population is evolved through several generations using GA with special genetic operators considering the acquired information. Two complementary processing levels are defined: Global Level and Local Level, each with its own independent tasks and processes. In the Global Level mainly GA process is carried out, whereas in the Local Level, conventional GNP based mining method is carried out in parallel and they generate their own local pools of association rules. Several special genetic operations for GA in the Global Level are proposed and the performance of each of them and their combination is shown and compared.In our simulations, the conventional GNP based mining method and our proposed method are compared using a real world large and dense database with a huge amount of attributes. The results show that extending the conventional GNP based mining method using GA allows to extract association rules from large and dense databases directly and more efficiently than the conventional GNP method.