Fejér-monotone hybrid steepest descent method for affinely constrained and composite convex minimization tasks

Abstract

ABSTRACTThis paper introduces the Fejér-monotone hybrid steepest descent method (FM-HSDM), a new member to the HSDM family of algorithms, for solving affinely constrained minimization tasks in real Hilbert spaces, where convex smooth and non-smooth losses compose the objective function. FM-HSDM offers sequences of estimates which converge weakly and, under certain hypotheses, strongly to solutions of the task at hand. In contrast to its HSDM's precursors, FM-HSDM enjoys Fejér monotonicity, the step-size parameter stays constant across iterations to promote convergence speed-ups of the sequence of estimates to a minimizer, while only Lipschitzian continuity, and not strong monotonicity, of the derivative of the smooth-loss function is needed to ensure convergence. FM-HSDM utilizes fixed-point theory, variational inequalities and affine-nonexpansive mappings to accommodate affine constraints in a more versatile way than state-of-the-art primal–dual techniques and the alternating direction method of multipliers do. Recursions can be tuned to score low computational footprints, well-suited for large-scale optimization tasks, without compromising convergence guarantees. Results on the rate of convergence to an optimal point are also presented. Finally, numerical tests on synthetic data are used to validate the theoretical findings.