In recent years, the innovation in devices and schemes of physical layer plays an important role in promoting the development of WDM-PON (Wavelength Division MultiplexingPassive Optical Network).Among the key technologies adopted in the WDM-PON system, advanced modulation formats at downstream and remodulations at upstream attract the most attention. DPSK (Differential Phase Shift Keying Modulation), FSK(Frequency-Shift Keying), IRZ(Inverted Return-to-Zero) and Manchester are four of the most common formats and each of them was studied before, but, however, nobody compared and analyzed them together.So, we will do the job by numeralsimulation in this paper. During our simulation, the 40-Gbit/s FSK is generated through the demodulation of two DPSK signals and 40-Gbit/s Manchester signal bases of employing a delay interferometer (DI) and an optical delay line as format conversion device to convert RZ-DPSK(Return to Zero-Differential Phase Shift Keying Modulation) coding to Manchester coding, both of the two format generation schemes are novel.Meanwhile,this paper emphasizes the input optical power and dispersion tolerance, which both related to the sensitivity of the system. Analyze the sensitivity with DPSK, FSK, IRZ and Manchester, and then compare the four advanced modulation formats.