Nowadays, the gap between urban and rural physical education (PE) in China is gradually widening. It is mainly manifested in the loss of PE teachers in rural areas. It is important to understand and solve the problem of the loss of rural PE teachers. The purpose of this paper is to explore this problem and propose some strategies to solve this problem. This study mainly used the following methods—literature review, logical analysis, and expert interview. In this study, we found the following reason of the loss of rural PE teachers: The salary and benefit package received by rural PE teachers tend to be low comparing with the rural teacher of other subject matters (e.g., Chinese, Mathematics). This is one major reason that results in the major loss of PE teachers in rural schools of China. Teachers’ living and working conditions in rural schools tend to be low comparing with the conditions in urban schools. PE teachers in rural schools tend to be assigned more responsibilities/work to do (e.g., teaching other classes like math, Chinese in addition to PE) comparing with urban teachers. The marginalization of PE in Chinese school system is another reason for the loss of PE teachers. PE teachers are more likely to be marginalized in rural schools, which greatly reduces PE teacher’s enthusiasm on their teaching career. Here are some strategies that we think can help solve, at least alleviate, the PE teacher loss problem: The government needs to enact national policies regarding to improving the salary and benefit package of teachers, especially PE teachers, in rural school; universities need to increase their enrollment of students majoring in PE, which tends to be able to increase the number of PE teachers; national policies are needed to address the unequal treatments of teachers in school, so that PE teachers can have the same benefits and responsibilities with teachers of other subject matters; more professional development opportunities should be provided for in-service teachers, so that PE teachers can further pursuit their teaching career and maintain their teaching enthusiasm; necessary facilities and equipment need to be provided for PE teachers to effectively teach PE class and annual fund should be provided to support the renew of equipment and teaching; schools should develop fair, effective, and reasonable teacher assessment system. Unqualified/ineffective teachers should be laid off, so that effective and enthusiastic teacher can have opportunities to be hired.