Question generation model based on key-phrase, context-free grammar, and Bloom’s taxonomy

Abstract

Automated question generation is a task to generate questions from structured or unstructured data. The increasing popularity of online learning in recent years has given momentum to automated question generation in education field for facilitating learning process, learning material retrieval, and computer-based testing. This paper report on the development of question generation framework based on key-phrase method for online learning with a constraint that the generated questions should comply with the learning outcomes and skills from Bloom’s Taxonomy. The proposed method was tested using learning materials of Software Engineering course for undergraduate level written in Bahasa Indonesia obtained from Bina Nusantara’s (Binus’s) Online Learning repository. Using one-semester lecture material, this study generated 92,608 essay-type questions from 6-level Bloom’s Taxonomy which were further sampled randomly to obtain 120 question samples for method evaluation. Performance evaluation using average Bilingual Evaluation Understudy (BLEU) involving five independent reviewers toward samples of these questions achieved 0.921 and 0.6 Cohen’s Kappa. The relevance of Bloom’s Taxonomy level of the generated questions was evaluated by means of classification model with 0.99 accuracy. The results indicate that not only are the generated questions well understood and agreed by the reviewers, they are also relevant to the expected Bloom’s Taxonomy level there for the questions can be delivered to students in the respected course delivery and evaluation.