This work conducts a systematic study of the impact of variable site and instrument conditions on the performance of a species recognition task. Echolocation clicks were collected from six different sites in the Southern California Bight from multiple deployments of instruments with nine different preamplifiers. The classification performance of a Gaussian mixture model using cepstral features is examined on Risso’s and Pacific White-Sided dolphins. One hundred three-fold Monte Carlo experiments are conducted. When grouped so that each acoustic encounter is either in the training or test set, a mean error rate of 1.9%±4.4σ is obtained. In spite of correction for preamplifier response curves, grouping by preamplifier and site increases error dramatically to 20.9%±18.1σ and 25.9%±28.1σ, respectively. We introduce noise compensation techniques that reduce error rates as follows: by encounter, 0.5%±0.3σ, by preamplifier, 1.7%±2.3σ, and by site, 9.4%±16.7σ.