Ecological forecasting models play an increasingly important role for managing natural resources and assessing our fundamental knowledge of processes driving ecological dynamics. As global environmental change pushes ecosystems beyond their historical conditions, the utility of these models may depend on their transferability to novel conditions. Because species interactions can alter resource use, timing of reproduction, and other aspects of a species' realized niche, changes in biotic conditions, which can arise from community reorganization events in response to environmental change, have the potential to impact model transferability. Using a long-term experiment on desert rodents, we assessed model transferability under novel biotic conditions to better understand the limitations of ecological forecasting. We show that ecological forecasts can be less accurate when the models generating them are transferred to novel biotic conditions and that the extent of model transferability can depend on the species being forecast. We also demonstrate the importance of incorporating uncertainty into forecast evaluation with transferred models generating less accurate and more uncertain forecasts. These results suggest that how a species perceives its competitive landscape can influence model transferability and that when uncertainties are properly accounted for, transferred models may still be appropriate for decision making. Assessing the extent of the transferability of forecasting models is a crucial step to increase our understanding of the limitations of ecological forecasts.