Learning-based digital back propagation to compensate for fiber nonlinearity considering self-phase and cross-phase modulation for wavelength-division multiplexed systems

Abstract

We propose a learning-based digital back propagation (LDBP) technique that addresses self-phase modulation (SPM) and cross-phase modulation (XPM) of a wavelength-division multiplexed (WDM) optical signal. The LDBP has a structure defined for each of the individual channels of a WDM signal, and connections between the channels address the XPM to effectively compensate for nonlinear waveform distortion of the signal in long-distance optical transmission systems. We attempt to optimize the limited number of parameters used in the structure such as the dispersion, nonlinear coefficient, and walkoff parameter to compensate for the nonlinear phase shift induced by the XPM. We derive equations to update the parameters through an iterative process based on the stochastic gradient descent algorithm. We verify the effectiveness of the proposed LDBP technique through a transmission experiment that uses an 11-channel WDM, 32-Gbaud, dual-polarization 16 quadrature amplitude modulation (QAM), and probabilistically shaped (PS) 64QAM signals. With the focus on the LDBP with a 1-step/span configuration, we operate the learning process using the received 16QAM signal in the experiment. We confirmed the successful convergence of particular parameters of the model of the transmission line after the developed learning procedure. We apply the LDBP with fixed optimized parameters to the received waveforms of the PS-64QAM signals and compare the performance with some DBPs. We observed that the proposed LDBP technique that considers XPM with 1-step/span configuration exhibits the best performance in compensating for nonlinear waveform distortion. Additionally, the learning process is effective for the case considering both SPM and XPM compared with the case of SPM only. Finally, we investigate the computational complexity of the LDBP and reveal that the total calculation cost is of the same order as that of a conventional DBP considering only SPM with a 2-step/span configuration.