Coordinate great circle descent algorithm with application to single-index models.

Abstract

Coordinate descent algorithm has been widely used to solve high dimensional optimization problems with a non-differentiable objective function recently. To provide theoretical justification, Tseng (2001) showed that it leads to a stationary point when the non-differentiable part of the objective function is separable. Motivated by the single index model, we consider optimization problems with a unit-norm constraint in this article. Because of this unit-norm constraint, the coordinate descent algorithm cannot be applied. In addition, non-separability of the non-differentiable part of the objective function makes the result of Tseng (2001) not directly applicable. In this paper, we propose a novel coordinate great circle descent algorithm to solve this family of optimization problems. The validity of the algorithm is justified both theoretically and via simulation studies. We also use the Boston housing data to illustrate this algorithm by applying it to fit single-index models.