The semiempirical Hamiltonians MNDO, AM1, PM3, RM1, PDDG/MNDO, PDDG/PM3, and SCC-DFTB, when used as part of a hybrid QM/MM scheme for the simulation of biological molecules, were compared on their abilities to reproduce experimental ensemble averages at or near room temperatures for the model system alanine dipeptide in water. Free energy surfaces in the (phi, psi) dihedral angle space, (3)J(H(N),H(alpha)) NMR dipolar coupling constants, basin populations, and peptide-water radial distribution functions (RDF) were calculated from replica exchange simulations and compared to both experiment and fully classical force field calculations using the Amber ff99SB force field. In contrast with the computational chemist's intuitive idea that the more expensive a method the better its accuracy, the ff99SB force field results were more accurate than most of the semiempirical methods, with the exception of RM1. None of the methods, however, was able to accurately reproduce the experimental data. Analysis of the results indicate that the specific QM/MM interactions have little influence on the sampling of free energy surfaces, and the differences are well explained simply by the intrinsic properties of the various QM methods.