The COVID-19 pandemic has driven substantial evolution of the SARS-CoV-2 virus, yielding subvariants that exhibit enhanced infectiousness in humans. However, this adaptive advantage may not universally extend to zoonotic transmission. In this work, we hypothesize that viral adaptations favoring animal hosts do not necessarily correlate with increased human infectivity. In addition, we consider the potential for gain-of-function mutations that could facilitate the virus’s rapid evolution in humans following adaptation in animal hosts. Specifically, we identify the SARS-CoV-2 receptor-binding domain (RBD) mutations that enhance human–animal cross-transmission. To this end, we construct a multitask deep learning model, MT-TopLap trained on multiple deep mutational scanning datasets, to accurately predict the binding free energy changes upon mutation for the RBD to ACE2 of various species, including humans, cats, bats, deer, and hamsters. By analyzing these changes, we identified key RBD mutations such as Q498H in SARS-CoV-2 and R493K in the BA.2 variant that are likely to increase the potential for human–animal cross-transmission.