Partially Degenerate Multi-objective Test Problems

Abstract

Degenerate multi-objective test problems are included in test suites to evaluate EMO algorithms on a wide variety of test problems. However, it was pointed out in some studies that the frequently-used degenerate DTLZ5, DTLZ6 and WFG3 test problems do not have degenerate Pareto fronts. Their Pareto fronts are different from the originally intended degenerate shapes. Actually, they are partially degenerate test problems. Modified formulations of DTLZ5 and DTLZ6 were proposed to remove the non-degenerate parts of their Pareto fronts. However, the original formulations of DTLZ5, DTLZ6 and WFG3 continue to be used as degenerate test problems in many studies whereas they are not degenerate test problems. One issue in their use as degenerate test problems is that reference point sets for IGD calculation are sampled from the originally intended degenerate Pareto fronts whereas they are not the true Pareto fronts. Nevertheless, the original DTLZ5, DTLZ6 and WFG3 formulations are useful for performance evaluation of EMO algorithms since their Pareto front shapes are similar to some real-world problems and much more complicated than other test problems. That is, their use helps us to evaluate the performance of EMO algorithms on a wide variety of test problems including realistic and challenging test problems. In this paper, we clearly demonstrate the usefulness of the original DTLZ5, DTLZ6 and WFG3 formulations. Then, after pointing out the difficulty in their use in computational experiments, we explain how we can obtain reliable experimental results on those test problems.