Swept source based coherent detection system (CDS) is a common method to measure weak signals in optical sensing and imaging systems. The nature of the phase noise of swept source based CDS however has attracted little attention. In this paper, the statistical properties of the phase noise in a CDS based on Fourier domain mode locked (FDML) laser are investigated theoretical and experimental. The fast spectral evolution of the FDML laser is characterized by using the time-varying phase. The probability density function (PDF) of the measured residual phase jitter (RPJ) is found to be close to the Gaussian distribution. The variance of the RPJ increases quadratically with the optical path difference of the CDS. The PDF of the RPJ and the variation of the variance of the RPJ and the frequency-noise power spectral density are studied numerically using stationary white noise. The experimental results agree well with simulations. The study provides insight into the noise property of FDML laser based CDS and evaluates the temporal coherence of swept lasers.