Breaking the Billion-Variable Barrier in Real-World Optimization Using a Customized Evolutionary Algorithm

Abstract

Despite three decades of intense studies of evolutionary computation (EC), researchers outside the EC community still have a general impression that EC methods are expensive and are not efficient in solving large-scale problems. In this paper, we consider a specific integer linear programming (ILP) problem which, although comes from a specific industry, is similar to many other practical resource allocation and assignment problems. Based on a population based evolutionary optimization framework, we develop a computationally fast method to arrive at a near-optimal solution repeatedly. Two popular softwares (glpk and CPLEX) are not able to handle around 300 and 2,000 integer variable version of the problem, respectively, even after running for several hours. Our proposed method is able to find a near-optimal solution in less than second on the same computer. Moreover, the main highlight of this study is that our method scales in a sub-quadratic computational complexity in handling 50,000 to one billion variables. We believe that this is the first time such a large-sized real-world constrained problem has ever been handled using any optimization algorithm. The study clearly demonstrates the reasons for such a fast and scale-up application of the proposed method. The work should remain as a successful case study of EC methods for years to come.