The inverse process of diffuse optical tomography (DOT) is ill-posed, which has a significant effect on the quality of the reconstructed results. In this paper, the polarization characteristics of light are introduced into the inverse solution model of DOT to increase the solution constraints and improve the imaging quality. The spectral element method is used to numerically solve the vector radiative transfer equation, which is used to simulate the propagation of polarized light in a scattering medium. The DOT reverse solution algorithm based on the polarization component is constructed so that the Stokes vector can be used for absorption coefficient and scattering coefficient reconstruction. The simulation and experimental results show that when polarized light is used for incidence, using the intensity component and the multi-polarization component to solve the DOT inverse model can improve the reconstruction accuracy of the medium absorption coefficient distribution and the DOT image quality.