Demonstration of effective global optimization techniques via comparative analysis on a large analytical problem set

Abstract

Many modern global optimization algorithms are inspired by natural phenomena rather than classes of mathematical functions. Theorems such as No Free Lunch imply that an algorithm's performance on an objective function is determined by the compatibility between the function and its structure. Grouping by inspiration blurs the distinction between algorithms, making it harder to study compatibility. Therefore, this work treats algorithms as sequential sampling algorithms, and groups them by sampling scheme: 1. perturb every design (e.g., Particle Swarm), 2. perturb a subset of designs (e.g., rand/1/bin Differential Evolution), 3. perturb a single design (e.g., best/2/bin Differential Evolution), 4. deterministically modify and then perturb the design (e.g., Quantum Particle Swarm). Using 295 analytical test cases, the structure and performance of 38 biologically inspired algorithms (major and minor variations of 5 algorithms) are compared by group. The groups are evaluated by 1. how performance scales with dimensionality, and 2. trends in mean convergence rates and accuracy. Controlling for sample size, number of algorithms/group, convergence criteria, and tuning parameters, Groups 2 and 3 demonstrate superior accuracy and convergence rates on 80 % of test cases combined, implying greater overall compatibility than other groups, and scale much better than other groups on 2nd and 4th order polynomials up to 100-dimensions, converging to minima 3---6 orders of magnitude lower. Statistical significance testing reveals overlap in the behavior of certain Group 2 and 3 algorithms on 52 test cases. Group 3 is a special case of Group 2, further implying structural compatibility with certain test cases.