Recurrent Neural Network Equalization for Wireline Communication Systems

Abstract

Satisfying the demand for ever increasing data rates in wireline communication systems is challenging due to high frequency-dependent channel insertion loss causing significant inter-symbol interference (ISI). This motivates the use of ADC-based receiver architectures that allow for digital equalizer implementations. However, the large tap count feed-forward equalizers (FFEs) required at high data rates causes noise enhancement that can degrade signal-to-noise ratio (SNR) and high complexity is required to implement multi-tap decision feedback equalizers (DFEs). Moreover, analog front-end (AFE) nonlinearity can limit the performance of these traditional FFE-DFE architectures. This brief investigates an alternative approach with recurrent neural network (RNN) equalization based on long short-term memory (LSTM) and gate recurrent unit (GRU) cells. 64 Gb/s PAM4 modeling results show that the RNN topologies offer significant BER improvement over a wide SNR range for a channel with 30 dB loss at 16 GHz and enable operation over a channel with a large notch at 10 GHz that the traditional FFE-DFE topology cannot support. Synthesis results utilizing a 22 nm FinFET process yield only a 34% and 29% increase in receiver power consumption for the LSTM and GRU implementations, respectively, to enable this improved performance.