Interpretation of the MLSD metric for time varying, frequency selective Rayleigh fading channels

Abstract

This paper describes a new interpretation of the maximum likelihood (ML) sequence detector (MLSD) for a linearly modulated signal transmitted over an unknown time varying, frequency selective, Rayleigh fading channel. The metric, which involves the channel autocovariance, is transformed into a more intuitive Mahalanobis distance (squared Euclidean distance in white noise) between the observations and a reference estimated from the observations. A novel estimator, called the autocovariance preserving estimator (APE), is obtained and its properties are characterised. As shown via an analytic example, its estimation bias and thus mean-squared estimation error are slightly worsened in order for the MLSD to attain reduced sequence error probability (SEP).