Effect of the number of samples used in a leave-one-out covariance estimator

Abstract

Some algorithms, such as Gaussian Maximum Likelihood require the use of the second order statistics, e.g. the covariance matrix, to help characterize the target in addition to the mean. Also, models such as FASSP require the second order statistics of targets to predict the performance of algorithms even though the algorithm may not use the covariance matrix directly. However, many times the number of samples available to make a good estimate of the covariance matrix is small. The Leave-One-Out Covariance (LOOC) estimator can be used to estimate the covariance matrix when the number of samples available is less than the normal minimum required. The normal minimum number of samples needed for a sample class covariance matrix is p+1 samples for p-dimensional data. For the LOOC estimator, in theory, as few as 3 samples are all that are needed. However, what are the affects of using such a low number in practice? This paper presents the results of an experiment that was conducted to measure what the affect may be in one specific instance. Sometimes as few as 0.1 p samples produce reasonably satisfactory results; other times 0.4p or more samples are needed.