Sensitivity of performance evaluation results by inverted generational distance to reference points

Abstract

The inverted generational distance (IGD) indicator has been frequently used for performance evaluation of many-objective algorithms. In this paper, we discuss the sensitivity of performance evaluation results by the IGD to the specification of a reference point set. Through computational experiments, we demonstrate that misleading evaluation results can be obtained by the use of the IGD. The reason for the misleading evaluation results is that the IGD tends to favor a solution set with a similar distribution to the reference point set. We demonstrate that such an undesirable bias of the IGD can be decreased by increasing the size of the reference point set and the size of solution sets to be compared. It is also shown that the bias can be decreased by using a modified IGD indicator called the inverted generational distance plus (IGD+). However, the bias becomes more severe by increasing the number of objectives. Our experimental results clearly demonstrate the necessity of very careful examination of performance comparison results by the IGD and IGD+ indicators.