The study aimed to investigate de-motivating factors in speaking in Indonesian EFL learning context and to propose solution to overcome speaking problems in teaching learning process. The description and discussion of findings are based on the results of a longitudinal mixed-method research in teaching Speaking Classes in an English Education Department at University in Indonesia. A qualitative inquiry was implemented to map the de-motivating factors faced by the EFL learners and Classroom Action Research (CAR) design was conducted to improve speaking ability. The results of the study reveal that there are fifteen de-motivating factors that impede the improvement of the speaking ability. The results of the intervention during the CAR cycles, cycle 1 and 2, prove that the implementation of Interest Based Instructional Materials (IBIM) improves the speaking ability. The implementation of IBIM encourages learners to activate their previous knowledge of instructional topics and build up their beliefs in presenting speaking tasks. The improvement in the speaking ability is affected by the role of teachers both during the pre-teaching phase and during the teaching learning process. The implication for the design and the development of instructional materials in Speaking Classes are discussed.