Due to inaccuracies in the modeling procedure, estimation errors, and poor data to parameter ratios, adaptation techniques can perform poorly when only a limited amount of data is available. Modeling inflexibility, on the other hand, limits their potential when large amounts of data are present. In this paper, we present a transformation-based Bayesian predictive approach to hidden Markov model (HMM) adaptation that addresses the above problems. The new technique, called Bayesian predictive adaptation (BPA), treats adaptation as model evolution arising from attempted transformation of the model parameters. The transformation is a structural representation of the assumed mismatch between the trained models and the adaptation data. Instead of estimating the transformation parameters directly, and blindly treating the estimates as if they are the true values, BPA averages over the variation of the parameters to generate a new model that can be used in the decoding process. By combining the power of Bayesian prediction to take into consideration the errors in estimation and modeling, with the power of transformation based techniques to use fewer parameters for adaptation, the proposed approach creates a new family of techniques that tend to be robust to estimation and modeling errors when only limited data are available, and to modeling inflexibility when large amounts of data are present. We present adaptation results under channel and speaker mismatches, and compare the performance of BPA to other adaptation techniques to demonstrate its effectiveness.