As a non-intellectual factor, Mathematics Adversity Quotient plays a crucial role in students’ learning of mathematics. This study adopts the modified mathematics adversity quotient table measuring left-behind and non-left-behind students’ Adversity Quotient in Chetian Minority Junior High School in Ziyuan, Guilin, Guangxi. It is found that the mathematics Adversity Quotient of non-left-behind junior high school students is generally higher than that of left-behind students, so there is a significant difference between them. Based on this situation, questionnaire survey and in-depth interview are adopted to explore the factors affecting the Adversity Quotient of left-behind junior high school students. Finally, it is concluded that home-school communication, teachers’ and parents’ care in school education have a significant influence on student’s Mathematics Adversity Quotient.