Mean-based Borda count for paradox-free comparisons of optimization algorithms

Abstract

Numerical comparisons are essential for selecting an efficient optimization algorithm for specific problems arising from various human practices. Note that recent researches have shown that paradoxes may occur for comparisons of the numerical performance of optimization algorithms, particularly the cycle-ranking paradox. Paradox-free is still open for some popular data analysis methods based on hypothesis testing (HT), which motivates us to design a class of HT-based paradox-free data analysis methods. The numerical comparison of optimization algorithms is analyzed for dimensional reduction in a matrix. The data collected during the experiment is stored in a four-dimensional matrix, which is then reduced to three-dimensional, two-dimensional, and finally a one-dimensional ranking vector. Then a mean-based Borda count (MeanBordaCount/T) is proposed to eliminate the cycle-ranking paradox that arises from the HT-based data analysis methods where HTs are performed on each problem. Specifically, hypothesis testing is replaced primarily by mean comparison, which has been proven that the result of hypothesis testing is the same as the result based only on mean comparisons, except that the former contains more equal or tied results. The Borda count is adopted to eliminate cycle-ranking in the final dimensional reduction. Finally, MeanBordaCount/T is proved to be the best choice among all HT-type methods, at least in the sense that it can minimize the error of pairwise comparisons.