Pareto Self-Paced Learning Based on Differential Evolution.

Abstract

Inspired by the learning rules of humans/animals, self-paced learning (SPL) ranks the samples from easy to complex and assigns real-valued weights to the samples. The current SPL regimes adopt an increasing pace parameter to select the training samples. Obviously, it is difficult to tune the pace parameter during iterations. Furthermore, the current SPL regimes cannot go back to the previous stage even when the current model is worse than the previous one. In this article, a novel Pareto SPL (PSPL) approach is proposed to address the above issues. In PSPL, the SPL problem is considered as a multiobjective optimization problem. Then, a Pareto-based differential evolution algorithm is utilized to optimize the self-paced function and the weighted loss function simultaneously. In PSPL, a new representation method is designed to assign nonpositive weights to the unselected samples. Then, a modified mutation operator inspired by the SPL idea is proposed to generate new population based on ranking the individuals and assigning weights. No pace parameters are introduced in the proposed technique and PSPL can obtain entire solution spectrum to automatically adjust the solution path. The effectiveness of the PSPL has been demonstrated through rigorous studies with matrix factorization and multiclass classification problems.