With the increasing demand for higher spectral efficiency in wireless communications, faster-than-Nyquist (FTN) signaling has been rediscovered to increase transmission rate without expanding signaling bandwidth. Most existing studies focus on low-complexity FTN receiver design by assuming perfect synchronization. In practice, however, phase noise (PHN) and carrier frequency offset (CFO) may degrade the performance of FTN detector significantly. In this paper, we develop iterative FTN detector in the presence of PHN and CFO in a factor graph framework. Wiener process is employed to model the time evolution of nonstationary channel phase. The colored noise imposed by sampling of FTN signaling is approximated by autoregressive model. Based on the factor graph constructed, messages are derived on the two subgraphs, i.e., PHN and CFO estimation subgraph and the FTN symbol detection subgraph. We propose two methods to update the messages between subgraphs, namely, Gaussian approximation via Kullback–Leibler divergence (KLD) minimization and the combined sum-product and variational message passing (SP-VMP), both of which enable low-complexity parametric message passing. The proposed SP-VMP algorithm can provide closed-form expressions for parameters updating. Moreover, conjugate gradient (CG) method is adopted to solve the maximum a posteriori probability (MAP) estimation of CFO with fast convergence speed. Simulation results show the superior performance of the proposed algorithm compared with the existing methods and verify the advantage of FTN signaling compared with the Nyquist counterpart.