Pairwise independence and its impact on Estimation of Distribution Algorithms

Abstract

Estimation of Distribution Algorithms (EDAs) were proposed as an alternative for traditional evolutionary algorithms in which reproduction operators could rely on information extracted from the population to enable a more effective search. Since information is usually represented as a probabilistic graphic model, the effectiveness of EDAs strongly depends on how accurately such models represent the population. In this sense, models of increasing complexity have been employed by EDAs, with the most successful ones being able to encode multivariate factorizations of joint probability distributions. However, some studies have shown that even multivariate EDAs fail to build accurate models for problems in which there is an intrinsic pairwise independence between variables. This study elucidates how pairwise independence impacts the linkage learning procedures of multivariate EDAs and affects their accuracy. First, the necessary conditions for learning additively separable functions are assessed, from which it is shown that extreme multimodality can induce pairwise independence. Second, it is demonstrated that in the presence of pairwise independence the approximate linkage learning procedures employed by many EDAs are not able to retrieve high-order dependences. Finally, in an attempt to infer how likely pairwise independence occur in practical problems, the case of non-separable functions is empirically investigated. For this purpose, the NK-model and the Linkage-Tree Genetic Algorithm (LTGA) were used as a study case and a range of usefulness for the LTGA was estimated according to N (problem size) and K (degree of interactions among variables and multimodality). The results indicated that LTGA linkage learning is probably more useful for K≤6 on instances with random linkages (this range grows with N), and for K≤9 on instances with nearest-neighbor linkages (this range is stable with N). Outside these ranges, pairwise independence is more likely to occur, which deteriorates models accuracy and impairs LTGA performance.