Feedback Categories Research Articles

Feedback in mathematics education research: a systematic literature review

ABSTRACT Research shows that feedback can have large positive effects on student achievement, but that effects vary considerably. Some of the variables that influence feedback’s effects are the characteristics of the feedback. When making decisions on which research to pursue, knowledge about the focus of previous research is useful. We conducted a systematic literature review of studies that address feedback in mathematics. The aim was to identify which feedback characteristics are addressed in the reviewed publications, which characteristics have been given ample research attention and which have been neglected. We searched for publications from 2012–2021 in three bibliographic databases: ERIC, Psychinfo, and Academic Search Premier. We identified 12 feedback categories and 5 variables which together describe the characteristics of the feedback in the publications found in the literature search. Feedback about the correctness of students’ answers and solution processes were common characteristics, while feedback on students’ self-regulation was uncommon.

Fostering self-regulated learning-focused feedback: Getting preservice teachers prepared

Since self-regulated learning (SRL) has been integrated into educational objectives, it is crucial for EFL teachers to incorporate SRL elements, such as feedback, into their instructional practices. Given the inherent complexity of the feedback process and the challenges associated with changing in-service teachers' feedback focus, developing preservice teachers’ literacy at the beginning of their professional careers is essential. To investigate the extent to which preservice teachers can generate multifocus feedback incorporating an SRL element, this study analyses the feedback categories generated by 286 preservice teachers and examines the underlying factors that influence their feedback choices using a mixed-methods approach. The feedback pattern analysis revealed that, despite recognizing the importance of SRL, preservice teachers struggled with creating multiple feedback entries supporting SRL. An investigation into SRL-focused feedback patterns and their influencing factors suggested several ways to embed SRL into the core of teacher feedback. These include building teacher beliefs to cultivate collegial resources, constructing a databank of feedback comments on SRL activities to facilitate feedback provision, and examining learner differences to craft individualized feedback. The conclusion summarizes the implications for future feedback studies, outlining an agenda for further research in this area.

Characteristics of positive feedback provided by UK health service users: content analysis of examples from two databases

BackgroundMost feedback received by health services is positive. Our systematic scoping review mapped all available empirical evidence for how positive patient feedback creates healthcare change. Most included papers did not...

A Language Model-Powered Simulated Patient With Automated Feedback for History Taking: Prospective Study.

Although history taking is fundamental for diagnosing medical conditions, teaching and providing feedback on the skill can be challenging due to resource constraints. Virtual simulated patients and web-based chatbots have thus emerged as educational tools, with recent advancements in artificial intelligence (AI) such as large language models (LLMs) enhancing their realism and potential to provide feedback. In our study, we aimed to evaluate the effectiveness of a Generative Pretrained Transformer (GPT) 4 model to provide structured feedback on medical students' performance in history taking with a simulated patient. We conducted a prospective study involving medical students performing history taking with a GPT-powered chatbot. To that end, we designed a chatbot to simulate patients' responses and provide immediate feedback on the comprehensiveness of the students' history taking. Students' interactions with the chatbot were analyzed, and feedback from the chatbot was compared with feedback from a human rater. We measured interrater reliability and performed a descriptive analysis to assess the quality of feedback. Most of the study's participants were in their third year of medical school. A total of 1894 question-answer pairs from 106 conversations were included in our analysis. GPT-4's role-play and responses were medically plausible in more than 99% of cases. Interrater reliability between GPT-4 and the human rater showed "almost perfect" agreement (Cohen κ=0.832). Less agreement (κ<0.6) detected for 8 out of 45 feedback categories highlighted topics about which the model's assessments were overly specific or diverged from human judgement. The GPT model was effective in providing structured feedback on history-taking dialogs provided by medical students. Although we unraveled some limitations regarding the specificity of feedback for certain feedback categories, the overall high agreement with human raters suggests that LLMs can be a valuable tool for medical education. Our findings, thus, advocate the careful integration of AI-driven feedback mechanisms in medical training and highlight important aspects when LLMs are used in that context.

Exploring the nature of peer feedback: An epistemic network analysis approach

AbstractBackground StudyPeer feedback has been used as an effective instructional strategy to enhance students' learning in higher education.ObjectivesThis paper reports on the findings of an explorative study that aimed to increase our understanding of the nature and role of peer feedback in the students' learning process in a computer‐supported collaborative learning (CSCL) setting. Exploring what types of feedback are used, and how they relate to each other and are related to academic performance has important implications for students and teachers.MethodsThis study was conducted in the higher education setting. It used a dataset consisting of student peer feedback messages (N = 2444) and grades from 231 students who participated in a large engineering course. Using qualitative methods, peer feedback was coded inductively. Epistemic network analysis (ENA) was used to analyse the relation between peer feedback types and performance.ResultsBased on the five types of peer feedback (i.e., ‘management’, ‘cognition’ ‘affect’, ‘interpersonal factors’ and ‘suggestions for improvements’), the results of the ENA showed that student feedback categories ‘management’, ‘cognition’ and ‘affect’ were positively related to student performance at the formative assessment phase.ConclusionsThe findings and the ENA visualizations also show that ‘suggestions for improvement’ and ‘interpersonal factors’ were not a significant part of student learning in peer assessment and feedback in the studied context.

Evaluating postgraduate family medicine supervisor feedback in registrars’ learning portfolios

Postgraduate supervision forms a vital component of decentralised family medicine training. While the components of effective supervisory feedback have been explored in high-income countries, how this construct is delivered in resource-constrained low- to middle-income countries has not been investigated adequately. This article evaluated supervisory feedback in family medicine registrars' learning portfolios (LPs) as captured in their learning plans and mini-Clinical Evaluation Exercise (mini-CEX) forms and whether the training district or the year of training affected the nature of the feedback. Registrars' LPs from 2020 across five decentralised sites affiliated with the University of the Witwatersrand in South Africa were analysed. Two modified tools were used to evaluate the quantity of the written feedback in 38 learning plans and 57 mini-CEX forms. Descriptive statistics, Fisher's exact and Wilcoxon rank-sum tests were used for analysis. Content analysis was used to derive counts of areas of feedback. Most learning plans (61.2%) did not refer to registrars' clinical knowledge or offer an improvement strategy (86.1%). The 'extent of supervisors' feedback' was rated as 'poor' (63.2%), with only 14.0% rated as 'good.' The 'some' and 'no' feedback categories in the mini-CEX competencies (p 0.001 top= 0.014) and the 'extent of supervisors' feedback' (p 0.001) were significantly associated with training district. Feedback focused less on clinical reasoning and negotiation skills. Supervisors should provide specific and constructive narrative feedback and an action plan to improve registrars' future performance.Contribution:Supervisory feedback in postgraduate family medicine training needs overall improvement to develop skilled family physicians.

Developmentally Moderated Focus on Form Instruction and Feedback in an Indonesian Kindergarten EFL programme

This study investigates the effect of developmentally moderated focus on form (DMFonF) instruction and feedback within processability theory (PT) on early grammatical development in an Indonesian kindergarten EFL programme, focusing on the acquisition of plural marking of nouns. The study has a pre-test, DMFonF intervention and post-test design. The participants were a teacher, and ten 1st-year and ten 2nd-year kindergarten children. Before the study, children received a meaning-based EFL programme. The children’s English development was measured based on PT stages pre- and post-test. Linguistic classroom environments before and during the DMFonF programme were analysed based on the DMFonF feedback categories. Regardless of how long they had been learning English, all the children were at the single-word stage at pre-test. They had all acquired lexical plural at post-test. Most of the children also acquired phrasal plural agreement. Additionally, the linguistic environment was much richer in DMFonF classes than in meaning-based classes.

AN OPEN ONLINE COURSE TO ENHANCE DIGITAL COMPETENCES OF TEACHERS

The purpose of this study was to design an online professional development program and evaluate the developed course in light of the implementation results. This course provided teachers with the opportunity to develop an ICT-enhanced lesson plan and apply it in their own classes. The current study was conducted using design-based research. Of the 171 teachers registered, 47 participated and 36 completed the full course and received certificates. The self-assessment of the 36 teachers on their ICT-enhanced classroom practices were used in the evaluation of the course. Second, teacher opinions were also solicited via the Teachers’ End-of-course Evaluation Questionnaire. Third, the course was evaluated using a rubric by the instructors. According to the results, the self-assessments of teachers on ICT-integration were high. The satisfaction of the teachers with the open online course was also very high, with 97% recommending the course. The course was evaluated as “exemplary” in terms of Learner Support & Resources, Online Organization & Design, and Instructional Design & Delivery categories, and as “effective” in the rubric’s Assessment & Evaluation of Student Learning, Innovative Teaching with Technology, and Faculty Use of Student Feedback categories. Future implementations could be revised by increasing interaction and feedback and providing additional implementation opportunities for the teachers.

A Study on Formative Feedback Categories for the Period for Integrated Studies

A Study on Formative Feedback Categories for the Period for Integrated Studies

Vuxna nybörjarstuderandes uppfattningar om skriftlig lärarrespons på skrivande i svenska som andraspråk

This study investigates how L2 beginner level students perceive written feedback from teachers on samples of their writing. It centers on beginner learners (of which very little research exists) and covers two broad areas of feedback: Focus and Manner. Focus refers to the kind of areas the teachers concentrate on in their comments, e.g. content, while Manner refers to different ways of delivering feedback, e.g. suggestions. The data was collected from a web-based course in Swedish for immigrants in an online Learning Management System. A questionnaire was used where the students evaluated feedback categories on a Likert scale and thereafter, ranked the same categories. Ranking, where different feedback categories are related to each other, has not been found in previous studies on feedback in the field of second language research. The results show that the students take feedback seriously. A large majority of students chose feedback on grammar as their first priority. They generally valued all feedback manners, but clearly preferred specific praise and specific criticism. I then discuss the conclusions and pedagogical implications based on web-based learning contexts. In particular, I emphasize the importance of establishing a dialogue between students and teachers which enables an orientation towards new understanding.

Supportive Peer Feedback in Tertiary Education: Analysis of Pre-Service Teachers’ Perceptions

To acquire knowledge about student-mediated peer-to-peer collaborative activities, pre-service teachers’ perceptions of peer feedback are analyzed and categorized as receiver, provider, or cognitive feedback. A questionnaire of 15 survey questions concerning supportive feedback from peers was designed and validated using assessments from more than 200 pre-service teachers. The questionnaire was aligned with the activities promoting supportive feedback between pre-service teachers from three bachelor’s degrees at a tertiary education institution. Their perceptions were then quantified in terms of the peer feedback categories. While there were significant correlations between the scores for all 15 questions, real insights were produced when the highest correlations were analyzed. As such, being involved as both feedback providers and receivers was highly rated. The self-efficacy of pre-service teachers receiving feedback, (i.e., the extent to which peer instructional strategies and the selected learning tasks were cognitively challenging so as to improve receiver feedback), proved to be correlated with their perceptions of involvement, autonomy, and structure. Likewise, motivation for providing or receiving feedback was also closely correlated with the self-efficacy of pre-service teachers providing feedback. Finally, all three questions in the cognitive feedback category were highly correlated. The pre-service teachers were, thus, motivated to improve their learning and considered feedback as a useful task and as a way to strengthen their relationships with their peers.

Comprehensive strategy for capturing and integrating community input into community research training curricula.

Community stakeholders often participate in community research training curricula development. There is limited information describing how their input informs curricula. This paper describes input solicitation methods, input received, and examples of its integration. From June 2014 to June 2016, community members (CMs) and community-based organizations (CBOs) guided curricula development tailored for CMs and CBOs, respectively. Engagement methods included a strategic planning retreat, surveys, a listening session, workgroup meetings, and community engagement studios. Descriptive statistics were used to summarize survey input. For other methods, input was extracted and compiled from facilitator notes. CMs (n = 37) and CBOs (n = 83) providing input included patients and caregivers and advocacy, community service, and faith-based organizations, respectively. The major feedback categories were training topic priorities, format (e.g., face-to-face vs. online), logistics (e.g., training frequency), and compensation (e.g., appro-priateness). Input directly guided design of CBO and CM curricula (e.g., additional time devoted to specific topics based on feedback) or helped to finalize logistics. Multiple quantitative and qualitative methods can be used to elicit input from community stakeholders to inform the development of community research training curricula. This input is essential for the development of training curricula that are culturally relevant and acceptable.

Patterns in the form of formative feedback and student response

Formative feedback currently receives attention as an effective means of increasing student learning. However, how to frame feedback to achieve the best effect is an ongoing debate. In this study we analyse a written data-set of 174 segments of teacher feedback and student response, coded using 10 emergent feedback and 14 response categories. As it is argued that feedback is a dialogue between students and teacher, links between feedback and response segments are viewed as a dialogical framework that we visualise and understand using network analysis. From this network we conclude that some ways of formulating feedback are more productive and likely to lead to types of responses that signify learning than others. We thus identify the reflection group of responses, consisting of the categories reflective response, explanation and students investigate own thinking. The feedback categories that link primarily to the reflection group are open question, wondering question and leading question, which we categorise as the questioning group of feedback. We discuss these patterns against a previously published framework, and by discussing specific examples we further our understanding of what makes feedback formative.

A categorisation of teacher feedback in the classroom: a field study on feedback based on routine classroom assessment in primary school

The aim of the present study was to examine and categorise teachers’ strategies for feedback in day-to-day communication in primary school. The different feedback categories constructed and grounded in data are applicable to feedback on learning and knowledge as well as on behavioural skills. Qualitative classroom observations were conducted in 4 primary school classrooms, including a total of 4 teachers and 75 students. A constructivist grounded theory approach was used throughout the analytical process. The analysis of the field data generated five main categories of feedback focuses: expecting, emotionally responding, normalising, steering and deliberating. The categories are all broad, yet with subcategories specific and nuanced, presenting concepts by which we can verbalise and communicate teachers’ feedback strategies. The categories place teachers’ feedback actions in a landscape, not on a linear axis. The complexity of feedback, as it is shown in the present study, challenges a dichotomisation of feedback and captures more of a complexity of classroom assessment.

A case study of using peer feedback in face-to-face and distance learning classes among pre-service teachers

This study investigated a feedback exchange activity for engaging pre-service teachers and the nature of such feedback in two undergraduate classes, a distance learning (DL) and a face-to-face (F2F) class. The research question asked if the nature of peer feedback was different between F2F and DL class formats. Students’ work samples were collected during class time to examine feedback function ratings. Participants also completed an online questionnaire to examine the nature of expectations of the feedback given. For both the nature of expectations and feedback function ratings, results for chi-square analyses demonstrated no statistically significant difference between the two modes of instruction (F2F versus DL), and the peer feedback categories. However, provision of feedback was significantly favoured in the F2F class over the DL class. Implications for course curriculum and future research are discussed.

A Study on Formative Feedback Categories for the Period for Integrated Studies

A Study on Formative Feedback Categories for the Period for Integrated Studies

Holistic perspective on Feedback for adult beginners in an online course of Swedish

Although several inquiries highlight the importance of feedback in language teaching and learning, there is a need for knowledge concerning a holistic perspective on feedback in the empirical context of written feedback for L2 adult beginners. The study reported here provides additional evidence about teachers’ actual feedback and student attitudes to feedback. The unit of analysis addresses a new context, namely Swedish as a second language, in an online course for adult beginners. The study included ten male and female university-level students with different cultural backgrounds. The purpose was to analyze several previously scientifically tested feedback categories for writing, which were conceptually replicated in the present study, as well as to additionally explore a new category for feedback on pronunciation in the same context. To establish causality, this study used attribution theory. The findings revealed top rankings for language accuracy and pronunciation in students’ evaluation. The teacher gave the most feedback on language accuracy. These results provide support for the importance of feedback on language accuracy which supports the empirical results of other inquiries. Furthermore, the study’s explorative findings support the need for further investigations on feedback on pronunciation. A proposition for future research is that more holistic type studies be conducted, including different categories and proficiency levels.

Understanding learners’ self-assessment and self-feedback on their foreign language speaking performance

This study examines university learners’ self-assessment and self-feedback on performance as captured in audio files from a foreign language speaking test. The learners’ were guided to listen, transcribe and analyse their own speaking samples, as well as propose future actions for improvement. Content of learners’ self-feedback was scrutinised against a feedback model, with data coded into various feedback categories as stipulated in the model for analysis. Results indicated that learners’ self-feedback was far reaching and multifaceted. Through self-feedback, learners’ identified discrepancies, answered feed up, feedback and feed forward questions, and inspected performance at task, process, self-regulation and self levels. Much of the feedback involved reflections on past learning history, other areas of learning, deviation of performance from preparation and learner personality traits. The self-feedback went largely beyond most teachers’ feedback capacity and bore great potential for learning and instruction. In particular, contrary to theoretical presumptions, self-level feedback seemed quite enlightening. Whether the observed quality self-feedback could actually help learners improve their performance, however, was not clear. It was suggested that some teachers’ time and effort be directed to the endeavour of facilitating learners’ self-assessment and self-feedback. Learners’ self-feedback capability should also be explored further in the development of relevant pedagogies and theories.

Analysis of New Zealand primary and secondary student peer- and self-assessment comments: applying Hattie and Timperley’s feedback model

Peer- and self-assessment (PASA) can lead to increased student self-regulation and achievement. However, few studies have examined the content of the feedback students in primary and secondary schools provide themselves and their peers. This study used Hattie and Timperley’s task, process, self-regulation and self feedback categories from their Review of Educational Research article ‘The power of feedback’ to (1) examine the content of a sample of naturally occurring student-written PASA comments (n = 471 utterances) and (2) explore the feasibility of using this model with student-generated feedback. Students provided primarily task feedback to both themselves and their peers, with self-regulation feedback only found in self-assessment. Students in higher grades tended to provide more task and process feedback, while giving less self feedback during self-assessment and more during peer assessment. More refinement of the model is recommended for both research and professional development purposes to better capture the quality and complexity of student-led feedback comments.

PS-158 Emergency Triage Assessment And Treatment (etat) Training In A Semirural Hospital In Sierra Leone – Retention Of Knowledge And Feed Back

Background and aims ETAT is a well established training course initiated through WHO in 2005. It is aimed at resource poor countries and teaches triage of children leading to effective emergency treatment. course is run over 3 days consisting of tutorials and practical skill stations. We delivered ETAT for the first time in a semirural hospital situated in Lunsar, Sierra Leone. Methods 20 candidates participated in the course. Candidates were asked to complete a test consisting of 25 points and to give structured feed back after completion of all modules. Results 20 initial post course assessments were performed with an average score of 78.2%. overall feedback was very good with all categories rated as excellent or good. Overriding themes were: The training should be repeated on a regular basis Tutorials and practical skill stations will help me to implement this knowledge into practice. Conclusions ETAT training is well received with all feedback categories rated as either excellent or good. initial testing demonstrated good knowledge of topics taught with an overall score of 78.2%. Post course retention of knowledge however is likely to be poor in line with previously published data. To achieve long term retention of knowledge ETAT training should be delivered repeatedly, ideally led by the local team.