Anxiety-trigerring conditions that can facilitate and debilitate learning must be made known to teachers. Besides, studies comparing foreign language anxiety in rural and urban areas are inconclusive and incongruous. Thus, this study is intended to explore learners’ language anxiety in speaking performance at urban and rural high schools. The investigation of language anxiety particularly addresses the extent of language anxiety, the anxiety factors as well as the relationship among language anxiety, school sites, and speaking. This study utilized a questionnaire, a test, and a learner interview to collect data by involving urban and rural school. Quantitative data analysis was carried out through descriptive statistics to find out learners’ language anxiety levels, and through Multiple Regression Correlation (MRC) to determine the relationship among the variables. Qualitative data analysis was conducted in relation to language anxiety sources by generating broad themes from the interview. The analysis revealed that (1) the extent of learners’ language anxiety was distinct from one another and subject to certain circumstances and their anxiety sensitivity; (2) four factors causing language anxiety were discovered; and (3) a new insight into the correlation between urban and rural learners, language anxiety, and speaking was noticeable. These findings suggest that teachers recognize the circumstances leading to anxiety, which is a basis for successful language acquisition and learning.