Performance limitations of an optical heterodyne continuous-phase frequency-shift keying transmission system affected by polarization mode dispersion in a single-mode fiber

Abstract

A theoretical analysis is developed to evaluate the conditional and average bit error rate (BER) performance degradation of an optical heterodyne continuous-phase frequency-shift keying (CPFSK) system caused by signal phase distortion due to polarization mode dispersion (PMD) in a single-mode fiber. The probability density function (pdf) of the random phase fluctuation due to PMD at the output of the receiver is determined analytically. The average BER performance results are evaluated at a bit rate of 10 Gb/s, considering Maxwellian distribution for the differential group delay (DGD) based on the pdf of the random phase fluctuation. The results show that the performance of the optical heterodyne CPFSK direct detection system suffers power penalties of 0.75, 1.95, 4.00, and 9.15 dB, corresponding to mean DGD of 20, 30, 40, and 60 ps, respectively, at a average BER of 10 -9 operating at 10 Gb/s. Furthermore, at increased values of the mean DGD, there occur BER floors above 10 -9 , which cannot be lowered by further increasing the signal power. It is noticed that BER floors occur at about 2×10 -8 , 10 -5 , and 2.5×10 -4 , corresponding to mean DGD of 60, 70, and 80 ps, respectively, at 10 Gb/s and a modulation index of 0.50. The effect of PMD is found to be more detrimental at higher modulation indexes and bit rates.