A nonlinear ANN equalizer with mini-batch gradient descent in 40Gbaud PAM-8 IM/DD system

Abstract

Abstract We propose and validate a novel nonlinear artificial neural network (ANN) equalizer for PAM-8 transmission in IM/DD system. Mini-batch gradient descent is introduced to efficiently train ANN equalizer. Using the proposed ANN equalizer, we successfully transmit a 40Gbaud PAM-8 signal over 4-km SMF with BER under the threshold of 3.8 × 10−3 and over 10-km SMF with BER under the threshold of 1 × 10−2. We also elaborately compare the proposed ANN equalizer with other methods including LMS equalizer, Volterra equalizer and look-up table (LUT). Experimental results indicate that ANN achieves the best performance that is slightly superior to Volterra equalizer with computational complexity half reduced. To the best of our knowledge, this is the first time to adopt mini-batch gradient descent to train ANN equalizer in IM/DD system.