Based on the hybrid frequency-domain (FD) and time-domain (TD) radiative transfer equation (RTE), a novel reconstruction strategy is developed to retrieve the optical parameter fields in the participating medium. The discrete ordinate method is employed to solve the forward FD and TD RTE to simulate the radiative transfer process. The sequential quadratic programming is utilized to solve the inverse problem for obtaining the optical parameters combined with an adjoint method. The Otsu algorithm is applied to the FD reconstructed results to distinguish the target region and the background region. To overcome the ill-posedness of the inverse problem, the generalized Gaussian Markov random field model is employed as the regularization term in the objective function. Various numerical experiments are studied and all results demonstrate that the proposed hybrid reconstruction scheme is suitable for efficient and accurate optical tomography by combining the FD and TD radiative transfer models. Meanwhile, the crosstalk of the ill-posed inverse problem can be alleviated effectively.