Computer-generated Feedback Research Articles

Second language articulatory training and computer-generated feedback in L2 pronunciation improvement

This paper investigates the efficacy of articulatory training and acoustic feedback on Persian L2 learners’ production of English segmental (/ɒ/). A sample of 30 Persian ESL learners was recruited- 10 learners were randomly assigned to the experimental group 1, 10 to the experimental group 2, and 10 to the control group. Over a five-week period, the experimental group 1 received training on the manner of articulation of the segment, the experimental group 2 received acoustic-articulatory training and was exposed to CALL software for receiving feedback, and the control group was only exposed to auditory input. The groups were given a pretest, an immediate posttest, and a generalization test. The results of the study showed a significant improvement in the performance of the participants in both the posttest and the generalization test in the experimental group 2. These findings suggest the inefficiency of the mere knowledge of the manner of articulation of the segment and lend support to the feasibility of using acoustic features of sounds and computer-based, learner-centred programs for second language segmental acquisition.

Student engagement with computer-generated feedback: a case study

In order to benefit from feedback on their writing, students need to engage effectively with it. This article reports a case study on student engagement with computer-generated feedback, known as automated writing evaluation (AWE) feedback, in an EFL context. Differing from previous studies that explored commercially available AWE programs, this study focused on a student’s engagement with a freely available AWE system. The construct of student engagement was operationalized in three interconnected dimensions: behavioural, emotional, and cognitive. Data were collected from three sources: ten written assignments completed by the student, AWE feedback on each assignment, and one retrospective interview with the student. Findings from this study suggest that AWE feedback is likely to have a positive impact on EFL writing, but this impact depends on how individual students engage with the feedback behaviourally, emotionally, and cognitively.

Developing Writing Skills in English Using Content-Specific Computer-Generated Feedback with EssayCritic

This paper presents a study of Norwegian Upper Secondary School students’ writing process in English with: 1) feedback from an essay critiquing system (EssayCritic) (target class) and 2) feedback from collaborating peers (comparison class). The students in both classes significantly improved their grades. In the target class, the feedback from EssayCritic gave content-specific cues and the students included more ideas in their essays than the students in the comparison class who struggled when giving feedback to each other.

The benefits of computer-generated feedback for mathematics problem solving

The goal of the current research was to better understand when and why feedback has positive effects on learning and to identify features of feedback that may improve its efficacy. In a randomized experiment, second-grade children received instruction on a correct problem-solving strategy and then solved a set of relevant problems. Children were assigned to receive no feedback, immediate feedback, or summative feedback from the computer. On a posttest the following day, feedback resulted in higher scores relative to no feedback for children who started with low prior knowledge. Immediate feedback was particularly effective, facilitating mastery of the material for children with both low and high prior knowledge. Results suggest that minimal computer-generated feedback can be a powerful form of guidance during problem solving.

People matter: Perceived sender identity modulates cerebral processing of socio-emotional language feedback

How do human brains integrate content with social context in communication? Recent research demonstrates that the perceived communicative embedding of perceptually identical language messages alters their cortical processing. When emotional trait-adjectives are perceived as human-generated personality feedback, event-related brain potentials are considerably larger than when the same adjectives are perceived as random computer-generated feedback. Here, we investigate the unique role of ascribed sender humanness for the underlying neural mechanisms. Participants were told that they were going to receive written positive, negative, or neutral feedback from an unknown stranger or from a socially intelligent computer system while high-density EEG was recorded. In the event-related potential (ERP), feedback from the ‘human sender’ elicited larger P2, Early Posterior Negativity (EPN), P3, and Late Positive Potential (LPP) components. The sources of this activity were localized in extended visual cortex, but also in the right superior frontal gyri, related to mentalizing about others, and the bilateral postcentral gyri implicated in embodied language processing. For emotional feedback, larger EPN, P3 and LPP amplitudes were also observed, resulting from enhanced activity in visual and temporal regions. Finally, for the EPN an interaction between sender and emotion was found, showing substantially increased visual processing of human-generated emotional feedback. These data confirm visual amplification effects induced by motivated attention but crucially also reveal distinct effects of perceiving a communication partner as human that activate ‘social brain’ structures. Obviously who is perceived as saying something can be as relevant as what is said and induce specific brain activity.

Effect of health risk assessment and counselling on health behaviour and survival in older people: a pragmatic randomised trial.

BackgroundPotentially avoidable risk factors continue to cause unnecessary disability and premature death in older people. Health risk assessment (HRA), a method successfully used in working-age populations, is a promising method for cost-effective health promotion and preventive care in older individuals, but the long-term effects of this approach are unknown. The objective of this study was to evaluate the effects of an innovative approach to HRA and counselling in older individuals for health behaviours, preventive care, and long-term survival.Methods and FindingsThis study was a pragmatic, single-centre randomised controlled clinical trial in community-dwelling individuals aged 65 y or older registered with one of 19 primary care physician (PCP) practices in a mixed rural and urban area in Switzerland. From November 2000 to January 2002, 874 participants were randomly allocated to the intervention and 1,410 to usual care. The intervention consisted of HRA based on self-administered questionnaires and individualised computer-generated feedback reports, combined with nurse and PCP counselling over a 2-y period. Primary outcomes were health behaviours and preventive care use at 2 y and all-cause mortality at 8 y. At baseline, participants in the intervention group had a mean ± standard deviation of 6.9 ± 3.7 risk factors (including unfavourable health behaviours, health and functional impairments, and social risk factors) and 4.3 ± 1.8 deficits in recommended preventive care. At 2 y, favourable health behaviours and use of preventive care were more frequent in the intervention than in the control group (based on z-statistics from generalised estimating equation models). For example, 70% compared to 62% were physically active (odds ratio 1.43, 95% CI 1.16–1.77, p = 0.001), and 66% compared to 59% had influenza vaccinations in the past year (odds ratio 1.35, 95% CI 1.09–1.66, p = 0.005). At 8 y, based on an intention-to-treat analysis, the estimated proportion alive was 77.9% in the intervention and 72.8% in the control group, for an absolute mortality difference of 4.9% (95% CI 1.3%–8.5%, p = 0.009; based on z-test for risk difference). The hazard ratio of death comparing intervention with control was 0.79 (95% CI 0.66–0.94, p = 0.009; based on Wald test from Cox regression model), and the number needed to receive the intervention to prevent one death was 21 (95% CI 12–79). The main limitations of the study include the single-site study design, the use of a brief self-administered questionnaire for 2-y outcome data collection, the unavailability of other long-term outcome data (e.g., functional status, nursing home admissions), and the availability of long-term follow-up data on mortality for analysis only in 2014.ConclusionsThis is the first trial to our knowledge demonstrating that a collaborative care model of HRA in community-dwelling older people not only results in better health behaviours and increased use of recommended preventive care interventions, but also improves survival. The intervention tested in our study may serve as a model of how to implement a relatively low-cost but effective programme of disease prevention and health promotion in older individuals.Trial RegistrationInternational Standard Randomized Controlled Trial Number: ISRCTN 28458424

Prism adaptation in virtual and natural contexts: Evidence for a flexible adaptive process.

Prism exposure when aiming at a visual target in a virtual condition (e.g., when the hand is represented by a video representation) produces no or only small adaptations (after-effects), whereas prism exposure in a natural condition produces large after-effects. Some researchers suggested that this difference may arise from distinct adaptive processes, but other studies suggested a unique process. The present study reconciled these conflicting interpretations. Forty participants were divided into two groups: One group used visual feedback of their hand (natural context), and the other group used computer-generated representational feedback (virtual context). Visual feedback during adaptation was concurrent or terminal. All participants underwent laterally displacing prism perturbation. The results showed that the after-effects were twice as large in the "natural context" than in the "virtual context". No significant differences were observed between the concurrent and terminal feedback conditions. The after-effects generalized to untested targets and workspace. These results suggest that prism adaptation in virtual and natural contexts involves the same process. The smaller after-effects in the virtual context suggest that the depth of adaptation is a function of the degree of convergence between the proprioceptive and visual information that arises from the hand.

Print-based self-help interventions for smoking cessation.

Many smokers give up smoking on their own, but materials giving advice and information may help them and increase the number who quit successfully. The aims of this review were to determine: the effectiveness of different forms of print-based self-help materials, compared with no treatment and with other minimal contact strategies; the effectiveness of adjuncts to print-based self help, such as computer-generated feedback, telephone hotlines and pharmacotherapy; and the effectiveness of approaches tailored to the individual compared with non-tailored materials. We searched the Cochrane Tobacco Addiction Group trials register. Date of the most recent search April 2014. We included randomized trials of smoking cessation with follow-up of at least six months, where at least one arm tested a print-based self-help intervention. We defined self help as structured programming for smokers trying to quit without intensive contact with a therapist. We extracted data in duplicate on the participants, the nature of the self-help materials, the amount of face-to-face contact given to intervention and to control conditions, outcome measures, method of randomization, and completeness of follow-up.The main outcome measure was abstinence from smoking after at least six months follow-up in people smoking at baseline. We used the most rigorous definition of abstinence in each trial, and biochemically validated rates when available. Where appropriate, we performed meta-analysis using a fixed-effect model. We identified 74 trials which met the inclusion criteria. Many study reports did not include sufficient detail to judge risk of bias for some domains. Twenty-eight studies (38%) were judged at high risk of bias for one or more domains but the overall risk of bias across all included studies was judged to be moderate, and unlikely to alter the conclusions.Thirty-four trials evaluated the effect of standard, non-tailored self-help materials. Pooling 11 of these trials in which there was no face-to-face contact and provision of structured self-help materials was compared to no intervention gave an estimate of benefit that just reached statistical significance (n = 13,241, risk ratio [RR] 1.19, 95% confidence interval [CI] 1.04 to 1.37). This analysis excluded two trials with strongly positive outcomes that introduced significant heterogeneity. Six further trials without face-to-face contact in which the control group received alternative written materials did not show evidence for an effect of the smoking self-help materials (n = 7023, RR 0.88, 95% CI 0.74 to 1.04). When these two subgroups were pooled, there was no longer evidence for a benefit of standard structured materials (n = 20,264, RR 1.06, 95% CI 0.95 to 1.18). We failed to find evidence of benefit from providing standard self-help materials when there was brief contact with all participants (5 trials, n = 3866, RR 1.17, 95% CI 0.96 to 1.42), or face-to-face advice for all participants (11 trials, n = 5365, RR 0.97, 95% CI 0.80 to 1.18).Thirty-one trials offered materials tailored for the characteristics of individual smokers, with controls receiving either no materials, or stage matched or non-tailored materials. Most of the trials used more than one mailing. Pooling these showed a benefit of tailored materials (n = 40,890, RR 1.28, 95% CI 1.18 to 1.37) with moderate heterogeneity (I² = 32%). The evidence is strongest for the subgroup of nine trials in which tailored materials were compared to no intervention (n = 13,437, RR 1.35, 95% CI 1.19 to 1.53), but also supports tailored materials as more helpful than standard materials. Part of this effect could be due to the additional contact or assessment required to obtain individual data, since the subgroup of 10 trials where the number of contacts was matched did not detect an effect (n = 11,024, RR 1.06, 95% CI 0.94 to 1.20). In two trials including a direct comparison between tailored materials and brief advice from a health care provider, there was no evidence of a difference, but confidence intervals were wide (n = 2992, RR 1.13, 95% CI 0.86 to 1.49).Only four studies evaluated self-help materials as an adjunct to nicotine replacement therapy, with no evidence of additional benefit (n = 2291, RR 1.05, 95% CI 0.88 to 1.25). A small number of other trials failed to detect benefits from using additional materials or targeted materials, or to find differences between different self-help programmes. Standard, print-based self-help materials increase quit rates compared to no intervention, but the effect is likely to be small. We did not find evidence that they have an additional benefit when used alongside other interventions such as advice from a healthcare professional, or nicotine replacement therapy. There is evidence that materials that are tailored for individual smokers are more effective than non-tailored materials, although the absolute size of effect is still small. Available evidence tested self-help interventions in high income countries; further research is needed to investigate their effect in contexts where more intensive support is not available.

The effects of computer-generated feedback on the quality of writing

Abstract This study provides a critical review of research into the effects of computer-generated feedback, known as automated writing evaluation (AWE), on the quality of students’ writing. An initial research survey revealed that only a relatively small number of studies have been carried out and that most of these studies have examined the effects of AWE feedback on measures of written production such as scores and error frequencies. The critical review of the findings for written production measures suggested that there is modest evidence that AWE feedback has a positive effect on the quality of the texts that students produce using AWE, and that as yet there is little evidence that the effects of AWE transfer to more general improvements in writing proficiency. Paucity of research, the mixed nature of research findings, heterogeneity of participants, contexts and designs, and methodological issues in some of the existing research were identified as factors that limit our ability to draw firm conclusions concerning the effectiveness of AWE feedback. The study provides recommendations for further AWE research, and in particular calls for more research that places emphasis on how AWE can be integrated effectively in the classroom to support writing instruction.

Assessment of student language skills in an e-learning environment

This article presents the role of various assessment structures that can be used in a VLE. e-Learning language courses offer tutors a wide range of traditional and computer-generated formative and summative assessment procedures and tools. They help to evaluate each student’s progress, monitor their activities and provide varied support, which comes from the tutor, the course structure and materials as well as other participants. The format and ownership of procedures are also very important as they introduce variety, comprehensiveness and personalisation into assessment. Besides computer-generated feedback and tutor marking, also peer assessment and self-assessment play a significant role because they help students build confidence and develop analytical, critical-thinking and reflection skills, which result in them becoming self-directed learners in the future. The introduction of e-assessment into course design can have a number of positive effects, but it may also cause a number of serious concerns. If appropriately applied, technology used for delivering formative feedback and evaluating student progress in a summative way can enhance the quality of the learning process. However, if an e-learning course is overloaded with computer-generated tests, it will not engage students, and the dropout rate will be high.

Learning not to feel: reshaping the resolution of tactile perception

We asked whether biased feedback during training could cause human subjects to lose perceptual acuity in a vibrotactile frequency discrimination task. Prior to training, we determined each subject's vibration frequency discrimination capacity on one fingertip, the Just Noticeable Difference (JND). Subjects then received 850 trials in which they performed a same/different judgment on two vibrations presented to that fingertip. They gained points whenever their judgment matched the computer-generated feedback on that trial. Feedback, however, was biased: the probability per trial of “same” feedback was drawn from a normal distribution with standard deviation twice as wide as the subject's JND. After training, the JND was significantly widened: stimulus pairs previously perceived as different were now perceived as the same. The widening of the JND extended to the untrained hand, indicating that the decrease in resolution originated in non-topographic brain regions. In sum, the acuity of subjects' sensory-perceptual systems shifted in order to match the feedback received during training.

A Comparison Of Team Performances Using Four Scenarios Of Feedback

When providing feedback to students working in groups, professors may use a variety of methods. Using computer generated feedback to give direction and correction for group assignments provides a way to maximize the professor’s time and provide consistent feedback across different groups and assignments. This article discusses the effect of two types of computer-generated feedback. The first is a semi-automated process for inserting pre-defined comments in student papers (the software is titled Strategy Assistant). This software allows the professor to define the most commonly used feedback information, then easily insert his comments on student papers. The second type is an automated process for giving grammar and style feedback. The automated grammar software (Edit Assist) checks the student paper for numerous grammar and style errors, then automatically inserts comments that identify problems. This article discusses the improvements in group performance with each of these types of feedback and measures the change in student performance during the course of a semester. Both types of feedback tended to improve student performance over the course of the semester, with the Strategy Assistant having the biggest impact on student grades and Edit Assist having the biggest impact on writing style error improvement.

Improving Temporal Bone Dissection Using Self‐Directed Virtual Reality Simulation

A significant benefit of virtual reality (VR) simulation is the ability to provide self-direct learning for trainees. This study aims to determine whether there are any differences in performance of cadaver temporal bone dissections between novices who received traditional teaching methods and those who received unsupervised self-directed learning in a VR temporal bone simulator. Randomized blinded control trial. Royal Victorian Eye and Ear Hospital. Twenty novice trainees. After receiving an hour lecture, participants were randomized into 2 groups to receive an additional 2 hours of training via traditional teaching methods or self-directed learning using a VR simulator with automated guidance. The simulation environment presented participants with structured training tasks, which were accompanied by real-time computer-generated feedback as well as real operative videos and photos. After the training, trainees were asked to perform a cortical mastoidectomy on a cadaveric temporal bone. The dissection was videotaped and assessed by 3 otologists blinded to participants' teaching group. The overall performance scores of the simulator-based training group were significantly higher than those of the traditional training group (67% vs 29%; P < .001), with an intraclass correlation coefficient of 0.93, indicating excellent interrater reliability. Using other assessments of performance, such as injury size, the VR simulator-based training group also performed better than the traditional group. This study indicates that self-directed learning on VR simulators can be used to improve performance on cadaver dissection in novice trainees compared with traditional teaching methods alone.

Improving Weight Loss Outcomes of Community Interventions by Incorporating Behavioral Strategies

We examined whether adding behavioral weight loss strategies could improve the outcomes of a community weight loss campaign. Shape Up RI is a 12-week, online, team-based program for health improvement in Rhode Island. In study 1, we randomly assigned participants to the standard Shape Up RI program or to the program plus video lessons on weight loss. In study 2, we randomly assigned participants to the standard program or to the program plus video lessons; daily self-monitoring of weight, eating, and exercise; and computer-generated feedback. Adding video lessons alone (study 1) did not result in significantly improved weight loss (2.0 ±2.8 kg vs 1.4 ±2.9 kg; P = .15). However, when the video lessons were supplemented with self-monitoring and feedback (study 2), the average weight loss more than doubled (3.5 ±3.8 kg vs 1.4 ±2.7 kg; P < .01), and the proportion of individuals achieving a weight loss of 5% or more tripled (40.5% vs 13.2%; P < .01). Participants in study 2 submitted self-monitoring records on 78% of days, and adherence was significantly related to outcome. Adding behavioral strategies to community campaigns may improve weight loss outcomes with minimal additional cost.

A balancing act: how can intelligent computer-generated feedback be provided in learner-to-learner interactions?

Task-based synchronous computer-mediated communication (CMC) may represent an optimal psycholinguistic environment for form-meaning connections, but learners do not receive feedback from a trusted authority. Intelligent computer-assisted language learning (ICALL) provides feedback, but the encouragement of communicative interaction remains largely unexplored. To combine the benefits of ICALL and CMC, we are designing a parser-based system that provides feedback on particle usage for first-year L2 Korean learners while they chat in CMC. Both to facilitate the use of a new orthography by beginning learners and to make processing feasible for the ICALL system, we guide the content of the activity by using picture-based information-gap tasks and a game record, and we control the range of allowable learner input by using a word bank. These constraints enable the system to address L2 errors in meaningful, goal-driven interactions.

A Methodology for Providing Individualised Computer-generated Feedback to Students

The traditional way of providing feedback to students after tests or assignments is labour-intensive. This paper explains the concepts and techniques used by the author to build computer-based applications that analyse students’ answers and generate individualised, detailed and constructive feedback. The paper explains how the data gathered from a student’s answers can be combined with other knowledge about the subject matter being taught, and the specific test questions, to create computerised routines that evaluate the individual student’s performance. This information can be presented in ways that help students to assess their progress, both in relation to their acquired knowledge in specified areas of study, and with regard to their ability to exercise relevant skills. In this way, appropriate feedback can be provided to large numbers of students quickly and efficiently. The same techniques can be used to provide information to the instructor about the performance of the group as a whole, with a degree of detail and accuracy that exceeds the impressions usually gained through traditional marking. The paper also explains the role of the subject instructor in designing and creating feedback-generating applications. The methodologies described provide insight into the details of the process and are a useful basis for further experimentation and development.

Verbal feedback from an expert is more effective than self-accessed feedback about motion efficiency in learning new surgical skills

Background Teaching of technical surgical skills to undergraduate medical students in a laboratory setting away from the patient is not common practice. Because of the large volume of students and shortage of available teaching faculty new methods of teaching must be developed for this group of trainees. In this study we examined the effectiveness of computer-based video training, different types of computer-based motion efficiency feedback (with and without expert criteria), and expert feedback on learning of a basic technical skill in medical students. Methods Forty-five junior medical students were randomized into 3 groups and learned suturing and knot-tying skills. Group A received computer-generated feedback about the economy of their movements. Group B received the same motion economy feedback, as well as expert reference values. Group C received verbal feedback from an expert. All groups were pre-tested, allowed 18 practice trials, and post-tested, and their skill retention was retested after 1 month. Performance was assessed by expert analysis using an objective structured analysis of technical skill and by computer analysis (Imperial College Surgical Assessment Device [ICSAD]). Results All groups showed improvement from pre-test to post-test. However, only group C showed retention of skill on delayed performance testing. Conclusions Verbal feedback from an expert instructor led to lasting improvements in technical skills performance. Providing information about motion efficiency did not lead to similar improvements.

A review of computer and Internet-based interventions for smoking behavior

This article reviews studies of computer and Internet-based interventions for smoking behavior, published between 1995 and August 2004. Following electronic and manual searches of the literature, 19 studies were identified that used automated systems for smoking prevention or cessation, and measured outcomes related to smoking behavior. Studies varied widely in methodology, intervention delivery, participant characteristics, follow-up period, and measurement of cessation. Of eligible studies, nine (47%) reported statistically significant or improved outcomes at the longest follow-up, relative to a comparison group. Few patterns emerged in terms of subject, design or intervention characteristics that led to positive outcomes. The “first generation” format, where participants were mailed computer-generated feedback reports, was the modal intervention format and the one most consistently associated with improved outcomes. Future studies will need to identify whether certain patients are more likely to benefit from such interventions, and which pharmacological and behavioral adjuncts can best promote cessation.

Self-efficacy Manipulation and State Anxiety Responses to Exercise in Low Active Women

The present study manipulated self-efficacy in an exercise context and examined its effect on the state anxiety of low active women. Participants ( N = 59) were randomly assigned to a low or high efficacy condition, and self-efficacy was manipulated by presentation of computer-generated false feedback after a graded exercise test. Participants returned for a second exercise bout several days later. Efficacy was successfully manipulated and participants in the high efficacy condition reported significantly less anxiety than those in the low efficacy condition both after the graded exercise test and before and after an acute bout of exercise. Results are discussed in terms of the effect of exercise intensity and manipulation condition on anxiety.

Promoting Self-Directed Learning in Simulation-Based Discovery Learning Environments Through Intelligent Support

Providing learners with computer-generated feedback on their learning process in simulation-based discovery environments cannot be based on a detailed model of the learning process due to the “open” character of discovery learning. This paper describes a method for generating adaptive feedback for discovery learning based on an “opportunistic” learning model that takes the current hypothesis of the learner and the experiments performed to test this hypothesis as input. The method was applied in a simulation–based learning environment in the physics domain of collisions. Additionally, the method was compared to an environment in which subjects received predefined feedback on their hypotheses, not taking the experimentation behavior into account. Results showed that overall both groups did not differ on knowledge acquired. A further analysis indicated that, in their learning processes, the learners in the experimental condition built upon their intuitive knowledge base, whereas the learners in the control condition built upon their conceptual knowledge base. In addition, measures of the learning process showed that the subjects in the experimental condition adopted a more inquiry-based learning strategy compared to the subjects in the control condition. We concluded, therefore, that providing learners with adaptive feedback had a different and beneficial effect on the learning process compared to more traditional predefined feedback.