Statistics on manifolds attracts more and more attentions because a variety of observations or responses have manifold structures. In this paper, we address the regression task on Grassmannian manifolds, which is arisen from the statistical shape analysis. Particularly, we develop an intrinsically semi-parametric model instead of the geodesic regression for better approximating shapes. Concretely, first we introduce a nonparametric term on Grassmannian manifolds to further depict the relationships that cannot be well described by the geodesic model on Grassmannian manifolds. Second, we utilize an alternatively iterative strategy to update parametric and nonparametric parts to form a solution. Finally, we testify the effectiveness of our regression model on synthetic data on with multiple combinations of n and m, as well as a real data of corpus callosum shapes. The experimental results validate that our proposed model outperforms the conventional geodesic regression model and shows the advantage of approaches driven doubly by model and data.