Automatic Morpheme-based Distractors Generation for Fill-in-the-Blank Questions using Listwise Learning-To-Rank Method for Agglutinative Language

Abstract

Automatic question generation facilitates the smart assessment for the evaluator to assess the student skills. Several methods were proposed to generate distractors for non-factoid cloze question using different similarity measures. This study presents a method for automatic generation of affix based distractor for Tamil fill-in-the-blank questions which are mainly used for learning Tamil grammar morphological details and vocabulary. In this study, affix based distractor generation is proposed as two step pipelined process: 1) Distractor candidate collection: This generation mainly relies on certain regularities manifest in high dimensional spaces which implicitly hybrids the orthographic and semantic features. 2) Distractor filtering: Filtering is trained as Learning-to-Rank models to persist the reliability in distractor generation. Feature based Listwise approaches (ListNet and ListMLE) were used which uses caserole relationship, subject-verb agreement, POS tag in addition to similarity measures. Experiments were done with annotated dataset (TamilMCQs) taken from 5th to 12th grade Tamil text books. Experimental results show that hybridization of spelling and semantic features highly improves the plausibility in distractor generation and then, ListMLE method improves the reliability in distractor generation compared to ListNet method. As a overall, our proposed pipelined process increases plausibility and reliability in distractor generation.