Abstract
We report on the design and pilot evaluation of a simple natural language computer tutor that targets student difficulties with the concepts of force and motion. The tutor prompts students to respond in free-response natural language to questions that address the relationships between the directions of net force, velocity, and acceleration. To examine the effectiveness of the natural language format, we compared student performance on a previously validated force and motion assessment after tutoring via natural language and multiple choice formats. Natural language training with feedback, multiple choice training with feedback, and natural language training without feedback formats resulted in effect sizes of d = 0.60 (p = 0.07), d = 0.46 (p = 0.13), and d = 0.09 (p = 0.97) respectively versus a no-training control. In addition, a median split on course grades showed no significant aptitude-treatment interaction across training conditions. However, accounting for time spent on training, the multiple choice training was significantly more efficient. For the natural language format, an analysis of performance (62% identification of an initial student response), false positives, and typical student answer patterns suggest room for improvement and subsequent study.
Highlights
The motivation behind the development of conversational computer tutors can be traced to the success of one-on-one tutoring methods
The mean-scores on the assessment are shown in Fig. 1 for each of the four training conditions: natural language format (68%), multiple choice format (66%), natural language format without feedback (57%), and the no-training control (54%)
There was no significant difference in scores between the natural language and multiple choice formats
Summary
The motivation behind the development of conversational computer tutors can be traced to the success of one-on-one tutoring methods. In particular, has an impressive pedigree of development – including ANDES/ATLAS [1], the AutoTutor series [2,3], Cordillera [4], and most recently Deep Tutor [5] These computer tutors, using various natural language methodologies and to significant levels of success, have tackled physics topics such as forces, kinematics, Newton’s laws, and energy conservation. The common hope is that by engaging the student in a reflective and constructive dialog, students will achieve greater learning gains than by rote application of physical principals In light of these successes, we chose to target a foundational subset of Newtonian Mechanics – a novel, systematic focus on the relationships between the directions of net force, velocity, and acceleration in one dimension. The precise use of language – or the ability to “talk like a physicist” is often heralded as one of many contentexternal goals of physics education
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
