This study examined the relationship between teachers’ expectations of performance and the academic learning time (ALT) of elementary students in physical education classes. Six teachers from different schools were asked to rank-order their students according to expected levels of performance in physical education. Thirty-six students, one boy and one girl from the high, middle, and low expectancy groups of each class, were chosen for observation in three basketball lessons to determine ALT–PE. Using ANOVA, a significant difference was found between level of teacher expectation and academic learning time of students. High expectancy students had significantly higher ALT–PE measures than middle expectancy students, who in turn had higher ALT–PE measures than low expectancy students. Significant differences in ALT–PE were also evident across the three basketball lessons observed. No significant difference was found between student gender and ALT–PE. These results suggest that a relationship does exist between teacher expectations and academic learning time in physical education.