Method for Optimizing Equalization in Nonlinear Signaling Analysis

Abstract

Nonlinear signaling analysis in nature is a high-dimensional optimization problem, the dimension of which can exceed 100 in today's high-speed digital and mixed-signal design. Optimizing the equalization (EQ) in such a high dimension is even more challenging. In this work, We present a fast method to optimize the equalization in nonlinear high-speed signaling. In this method, the nonlinear responses in the high-dimensional space due to all EQ settings are represented by a rank- $k$ model, where $k$ is in hundreds, which is many orders of magnitude smaller than the original problem dimension. From the rank- $k$ model, we recover the signature responses of each EQ, from which the best EQ setting is determined. Numerical simulations of large-scale real-world nonlinear channels for high-speed signaling have demonstrated the performance of the proposed method.