The output signal of the fiber optic gyro (FOG) often contains a large amount of noise, which makes it difficult to model and calibrate. Therefore, it must be denoised. In this paper, an adaptive noise cancellation method based on variational mode decomposition (VMD) is proposed. The FOG signal was decomposed into a series of mode components with different center frequencies, and the correlation coefficient method was used to determine the number of decomposition layer. The low-frequency signal and the high-frequency noise components were distinguished by the frequency distribution of each component. They were respectively accumulated to use as the ready denoising signal and the reference noise, then obtaining the denoised signal by adaptive noise cancellation. Using FOG measured data, the proposed method was taken into experiment compared with traditional methods. Allan variance analysis verified that the proposed method effectively suppresses the noise in the FOG signal.