A new formalization of minimum classification error using a Parzen estimate of classification chance

Abstract

Ina previous work, we showed that the minimum classification error (MCE) criterion function commonly used for discriminative design of pattern recognition systems is equivalent to a Parzen window based estimate of the theoretical classification risk. In this analysis, each training token is mapped to the center of a Parzen kernel in the domain of a suitably defined random variable; the kernels are then summed and integrated over the domain of incorrect classifications, yielding the risk estimate. Here, we deepen this approach by applying Parzen estimation at an earlier stage of the overall definition of classification risk. Specifically, the new analysis uses all incorrect categories, not just the single best incorrect category, in deriving a correctness function that is a simple multiple integral of a Parzen kernel over the region of correct classifications. The width of the Parzen kernel determines how many competing categories to use in optimizing the resulting overall risk estimate. This analysis uses the classic Parzen estimation method to support the notion that using multiple competing categories in discriminative training is a type of smoothing that enhances generalization to unseen data.