English is mandatory for Malaysian students from standard one until form five. The subject per se has been one of the prerequisites to pass Sijil Pelajaran Malaysia (SPM) and to attain a high school certificate. Regardless of the long years formally learning and acquiring the language within the classroom walls especially, many still fail to speak fluently. Furthermore, the issue of low English proficiency among Malaysians has been one of the primal causes of unemployment in the country. In terms of an individual’s speaking performance, it is found that anxiety during speaking a particular language can cause mental barriers that will prevent learners from speaking fluently. This quantitative study examined the levels of speaking anxiety of Form 1 students towards the English language and the contributing factors towards English-speaking anxiety. 138 Form 1 students took part and completed an adapted 33-item English Language Classroom Anxiety Scale (ELCAS) questionnaire. The questionnaire was then analyzed using Statistical Package for the Social Sciences (SPSS) for the mean and frequency values. The results show that 56.5% of the Form 1 students have moderate levels of English-speaking anxiety. It is also found that the highest contributing factor towards the students’ speaking anxiety is Fear of Negative Evaluation (FNE) with a mean of 2.59. In conclusion, English proficiency level, and socioeconomic as well as geographical factors influence the level of speaking anxiety among ESL learners in Malaysia.