A combination of optical frequency division multiplexing (FDM) and phase-shift-keying (PSK) homodyne detection can increase transmission capacity. However, phase sensitive transmission systems, especially repeatered ones, suffer from data-dependent optical amplitude fluctuation that is converted to phase fluctuation by fiber nonlinearity. The authors discuss how this data-dependent amplitude fluctuation affects the error rate performance of optical FDM PSK homodyne detection systems. If only the optical amplitude fluctuation induced by phase modulators is taken into account, the allowable power fluctuation to keep the power penalty at 0.5 dB at a bit error rate (BER) of 10/sup -10/ is below 0.17 mW for BPSK homodyne detection and 0.09 mW for QPSK homodyne detection. However, if only the amplitude fluctuation induced by the fiber chromatic dispersion is taken into account, the allowable number of repeaters to keep a 0.5-dB power penalty due to XPM at a BER of 10/sup -10/ is 1 for BPSK homodyne detection and below 5 for QPSK homodyne detection.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>