Regularization on a rapidly varying manifold

Abstract

The presence of inflection points on data manifold or rapid variation makes it difficult for the second order derivative based graph Laplacian and Hessian regularization techniques to accurately approximate the marginal distribution parameters. Moreover, in general, function over-fitting on seen unlabeled instances due to lack of extrapolation power which makes graph Laplacian regularization based solution biased towards constant. Hessian solves this problem by opting a generic function based on the function’s divergence in more than one direction. However, due to the presence of inflection points in the dense region, the function remains unpenalized by Hessian manifold regularization. We propose a Jerk based manifold regularization (JR) for dense, oscillating and manifolds with inflection points. JR approximates the rate of change of curvature from the underlying manifold which appropriately identifies the unpenalized geodesic deviating functions and accurately penalizes them. It also helps in identifying the optimal function in the presence of inflection points. Extensive experiments on synthetic and real-world datasets show that the proposed JR technique approximates accurate and generic input space geometrical constraints to outperform existing state-of-the-art manifold regularization techniques by a significant margin.