A class of feature extraction criteria and its relation to the Bayes risk estimate

Abstract

Feature extraction criteria of the form <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f(D_{l}, \cdots ,D_{M},\Sigma_{1}, \cdots , \Sigma_{M})</tex> are considered where <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D_{i}</tex> and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">\Sigma_{i}</tex> are the conditional means and covariances. The function <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f</tex> is assumed to be invariant under nonsingular linear transformations and coordinate shifts. For the case <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f(D_{1},D_{2},\Sigma_{o})</tex> , <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f</tex> is shown to depend only upon the distance <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">(D_{2}-D_{1})^{T} \Sigma_{o}^{-1}(D_{2}-D_{1})</tex> between classes. The <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</tex> class case, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f(D_{1}, \cdots ,D_{M}, \Sigma_{o})</tex> , is shown to depend upon the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">(M-1)M/2</tex> between-class distances <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">(D_{j}-D_{k})^{T} \Sigma^{-1}_{O}(D_{j}-D_{k})</tex> . This criterion is also shown to be equivalent to the mean-square-error of the general Bayes risk estimate. The most general <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f</tex> is reduced to a function of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</tex> sets of between-class distances with metrics induced by the conditional covariances. When <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M=2</tex> this dependence is reduced to two parameters which may be regarded as different between-class distance measures. Finally, the linear mapping is reduced to a one-parameter problem as in the work of Peterson and Mattson.