Nonlinearity mitigation with a perturbation based neural network receiver

Abstract

We propose a less complex neural network that estimates and equalizes the nonlinear distortion of single frequency dual polarization data transmitted through a single mode optical fiber. We then analyze the influence of the size of the input data symbol window on the neural network design and the enhancement of the quality factor (Q-factor) that can be achieved by integrating the neural network with a perturbative nonlinearity compensation model. We significantly reduce the complexity of the neural network by determining the most significant inputs for the neural network from the self-phase modulation terms (intra-cross phase modulation and intra-four wave mixing) in the model. The weight matrices of the neural network are determined without prior knowledge of the system parameters while the complexity of the network is reduced in two stages through weight trimming technique and principle component analysis (PCA). Applying our procedure to a 3200 km double polarization 16-QAM optical system yields a ≈ 0.85 dB Q-factor enhancement with a 35% smaller number of inputs compared to previous designs.