A reduced state variant of maximum likelihood sequence detection attaining optimum performance for high signal-to-noise ratios

Abstract

In the detection of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m</tex> -level pulse-amplitude modulation (PAM) signals transmitted over a noisy linear <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">\nu</tex> -symbol memory channel, maximum likelihood sequence detection (MLSD) is asymptotically optimum in the sense that the exponent of the probability of symbol error in the limit of small noise is the largest that can be achieved by any detector. A reduced state detection (RSD) is developed with the aim of mitigating the computational complexity of MLSD while attaining the same error exponent as MLSD. In RSD, for each baud, the recent received signal levels delineate a certain list of the most probable of the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m^{nu}</tex> possible states. Retention of only the paths threading through these most probable states is the key to success of RSD. Suppose it is required that the number of states retained at any time not exceed <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">\psi</tex> . When <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">\psi</tex> is less than the number of states required for RSD, it is shown how to modify RSD so that no more than <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">\psi</tex> states are retained and how to assess the degradation of the error exponent.