Small sample regression: Modeling with insufficient data

Abstract

Modeling with a small set of samples will normally result in great variance. This research proposes a unique procedure for small sample regression systematically using the concept of robust Bayesian inference and a contamination prior. The approach enlarges the possible domain of population information and attempts to estimate regression parameters. A data augmentation step included in the procedure is devoted to enlarging the original small data set by adding new data to the original data set. It follows that when the expectation-maximization (EM) algorithm is used for outputting the hypothesis h=〈β,σ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> 〉, approximating the true (but unobservable) β <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">*</sup> and σ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2*</sup> based on the enlarged data set. Both the augmented data set and the used maximum likelihood estimate are generated from contaminated priors. The experiments provided herein exhibit that the proposed procedure can effectively lower mean squared error when modeling.