In this paper, we obtain the weak convergence of functionals of non-homogeneous Markov chain. Our limiting processes are stochastic integrals. As applications, we first consider the unit root testing problem, which involves the weak convergence to stochastic integrals. Furthermore, we study the asymptotic distribution of least squares estimator in cointegrating regression model which involves endogeneity, nonstationary and nonlinearity, which depends heavily on weak convergence to stochastic integral.