This paper studies a new task of generating commonsense reasoning questions (QG) with diversity and controlled difficulty based on the given text. Unlike existing shallow questions, ours are more complex, in-depth, and thought-provoking. The answers need to be inferred from multiple clues in the text context. Some clues are implicit and require to be deduced from external commonsense knowledge. This process involves a series of comprehension skills, such as complex reasoning, hidden knowledge inference, etc. That can support many applications in the field of QA, online education, etc. Existing work mostly learns a binary mapping QG model, which is hard to control diversity and difficulty, leading to shallow results. Also, this model relies heavily on lots of scarce training data, which is unsuited to the low-resource scenario. To address these problems, we propose a new controllable framework to yield diverse in-depth results with low resources. In detail, we first build a graph to capture potential contextual and commonsense clues. To improve controllability, we utilize a reasoning skeleton with a desired difficulty level to retrieve a question-specific subgraph. It embodies all the necessary clues that can guide the asking direction well. The classical asking skeletons and expressive forms are learned from unlabeled data which is easy to obtain. The forms contain useful syntactic structures that help to organize questions well. By multiple sampling of the forms, we can yield diverse results. We optimize the results’ quality by multiple reinforced rewards, including context consistency, reasoning difficulty, and fluency. Moreover, we employ the generated results to facilitate the downstream application of fact-checking. Extensive experiments are conducted on two popular datasets. The results verify the effectiveness of our new approach, with 11% performance improvements on average.
Read full abstract