Feedback Tool Research Articles

Peer editing and writing proficiency: an experimental study in South Korean higher education

ABSTRACT Peer editing enhances critical thinking, improves student writing, ultimately leading to a more productive learning environment (Kostopoulou and O’Dwyer 2021; Lundstrom and Baker 2009, “To give is better than to receive: The benefits of peer review to the reviewer’s own writing.” Journal of Second Language Writing 18:30–43; Storch 2018, “Written corrective feedback from sociocultural theoretical perspectives: A research agenda.” Language Teaching 51 (02): 262–277; Yao et al. 2021, “How university EFL writers’ beliefs in writing ability impact their perceptions of peer assessment: Perspectives from implicit theories of intelligence.” Assessment & Evaluation in Higher Education 46 (1): 151–167). However, reservations toward peer feedback in a university EFL setting persist due to the accuracy of L2 feedback (Braine, 2003; Erdel 2023, “Student feedback literacy (SFL) profiles in an L2 writing classroom and the influence of peer feedback on the SFL features.” International Journal of Education and Literacy Studies 11 (3): 112–125.). This study utilized a pretest-posttest writing assessment to analyze differences in writing outcomes in an academic research writing course at two universities in Seoul, South Korea. The researchers also examined the accuracy of the L2 peer-editing feedback. The sampling consisted of five randomized classes of university undergraduates. Two control groups received feedback via the instructor and two experimental groups received peer-editing feedback. A fifth mixed group was divided between the control and experimental groups but within the same context. The instructors used both in-class and online feedback tools for the control and experimental groups. The analysis used Sengupta’s (1998, “Peer evaluation: ‘I am not the teacher’.” ELT Journal 52 (1): 19–28) revision taxonomy to assess peer feedback accuracy. Participants completed separate pretest and posttest writing assessments. One at the start of the semester (pretest) and one at the end (posttest). The researchers used a MANOVA to compare the pretest and posttest mean average assessment scores between the control and experimental groups. The results show that peer feedback matches the effectiveness of professor feedback for pretest/posttest scores. Researchers also assessed 156 peer-editing assignments for feedback accuracy. Peer editors demonstrated statistically significant accuracy. This study recommends using peer feedback for its benefits to L2 learners and instructors.

Crowd-Sourced Assessment of Laparoscopic Surgical Skills for Ob/Gyn Residents

Importance Feedback is an essential part of graduate medical education. With increasing demands on time and fewer opportunities for formal feedback in residency, there is an increasing need for objective and actionable feedback. Objective To understand how the Crowd-Sourced Assessment of Technical Skills (CSATS) assessment of residents’ laparoscopic skills compares with the current faculty assessment in an obstetrics/gynecology residency program. Design, Setting, and Participants This was a prospective educational intervention study involving residents from a single academic obstetrics/gynecology residency program at a large tertiary teaching hospital. Individual performance videos of hysterectomies, myomectomies, and endometriosis resections were recorded and uploaded to the CSATS database where they were analyzed. Formal written feedback reports from the CSATS system were sent to the residents. Residents then completed a survey addressing satisfaction, timeliness, and utility of the CSATS feedback method overall and in comparison with the residency’s in-person feedback and with the standard written, electronic feedback evaluation. Results A total of 16 residents submitted 65 videos of operations. Overall comparison of CSATS vs faculty evaluation was equivalent. Residents found that CSATS response time was slower than receiving in-person feedback from faculty. When comparing CSATS with the standard feedback form, residents felt strongly that CSATS was superior for both surgical skill feedback and procedure evaluation, with both categories having a mean top score. When comparing low-volume CSATS utilization vs high-volume CSATS utilization, there were no statistical differences. Overall, residents reported high satisfaction with CSATS evaluation. Conclusions Residents were generally satisfied with CSATS as an overall feedback tool. CSATS could be a possible addition to help improve resident education.

Cocreation of a Video Feedback Tool for Managing Self-Care at Home With Pairs of Older Adults: Remote Experience-Based Co-Design Study.

Involving older adults in co-design processes is essential in developing digital technologies and health care solutions to enhance self-care management at home, especially for older adults with chronic illness and their companions. Remote co-design approaches could provide technologically sustainable solutions that address their personal needs. This study aimed to cocreate and test the usability of a video feedback tool to facilitate self-care management at home. This experience-based co-design approach involved collaboration between 4 pairs of older adults, 4 researchers, and 2 service designers in three steps: (1) six iterative workshops (5 remote and 1 in person) to cocreate self-care exercises within an existing video feedback tool by identifying factors influencing self-care management; (2) developing and refining the self-care exercises based on suggestions from the older adults; and (3) usability testing of the cocreated exercises with the 4 pairs of older adults in their homes. Among the older adults (68-78 years), 3 adults had heart failure and 1 adult had hypertension. Data were analyzed inductively through thematic analysis and deductively using the USABILITY (Use of Technology to Engage in Adaptation by Older Adults and/or Those With Low or Limited Literacy) framework. The identified influencing factors guiding the contents and format development of 2 new self-care exercises were that pairs of older adults support and learn from each other in performing self-care, which increases their motivation and engagement in practicing self-care at home. The usability test of the 2 new self-care exercises, "Breathing exercises" and "Picking up from the floor," revealed that the pairs found the exercises and the video feedback component valuable for learning and understanding, for example, by comparison of performances highlighting movement variability. However, they found it difficult to manage the video feedback tool on their own, and a support structure or tailored education or training was requested. This study emphasizes that the video feedback tool holds the potential to facilitate learning and understanding in self-care management, which may support motivation. The studied video feedback tool can be beneficial for pairs of older adults managing self-care at home as a complement to traditional health care services, but an accurate supporting structure is required. The effectiveness of the video feedback tool and its integration into existing health care services still need to be assessed and improved through careful design and structured support.

Bridging the gap: a qualitative process evaluation from the perspectives of healthcare professionals of an audit-and-feedback-based intervention to improve transition to adult care for young people living with type 1 diabetes

BackgroundThe transition from pediatric to adult care is a vulnerable time for young people living with type 1 diabetes (T1D). Bridging the Gap (BTG) is an audit-and-feedback (AF) intervention aimed at improving both transitions-in-care processes and diabetes management in the year following transition. As part of BTG, we conducted a qualitative process evaluation to understand: (a) what was implemented and how; and (b) the contextual factors (micro-, meso- and macro-) that affected implementation, outcomes and study processes.MethodsUsing qualitative descriptive methodology, interviews were conducted with 13 healthcare professionals (HCPs) delivering diabetes care to transitioning youth. Participants were asked about their experiences of BTG study processes and feedback tools, the quality improvement (QI) initiatives implemented at their site, and potential spread and scale. Interviews also explored the impacts of COVID-19 on transition care and study processes and results.ResultsFive key themes were identified. Participants’ reflections on the BTG study design indicated they appreciated its flexible, site-specific approach to QI, which they saw as crucial to the success of their initiatives. Engagement with feedback reports and other study resources provided comparative, site-specific data. Participants described the challenges posed by the COVID-19 pandemic and its impacts on patients, care provision and study implementation. Their site-specific QI initiatives resulted in changes to their transition practices. Finally, participants commented on how BTG and its processes fostered a community of practice (CoP) between sites, resulting in new opportunities to collaborate and share experiences.ConclusionsBTG resulted in a CoP among practitioners delivering transition care to youth with T1D, which could be scaled up to promote a learning health system in pediatric diabetes care. Qualitative process evaluation is a useful tool for understanding how contextual factors affect the implementation and outcomes of complex QI interventions.

SỬ DỤNG GOOGLE DOCS VÀ MERCI APP TRONG LỚP HỌC VIẾT TIẾNG PHÁP

This study compares the effectiveness of two automated feedback tools, Google Documents, which will be further addressed as Google Docs, and Merci App, in improving writing ability in French as a foreign language. Forty second-year students used these two reviewer tools to correct their B2-level texts. The data includes 80 student drafts edited on two tools and a survey questionnaire after using these two applications. The results show that both tools help students correct many errors at the sentence, word, and fragment levels but cannot correct errors at the inter-sentence, idea, and coherence levels. Google Docs detects more grammatical errors than Merci App, but Merci App is superior when it comes to finding errors in spelling, vocabulary, expression, and punctuation. Students use the application's error correction suggestions to correct their papers at a very high rate, 92.6% with Google Docs and 81.7% with Merci App. The number of students who prefer Google Docs is also higher than the number of students who prefer Merci App. The study proposes to use both tools to correct students' writing errors and to help students become familiar with error explanations in Merci App to help them systematize language knowledge and develop auto-learning capacity.

Students Can Identify Quality Teachers, but can They Distinguish Between Dimensions of Quality Teaching? a Comparative Analysis of the Structure Behind the Tripod Survey

ABSTRACT Student perception surveys are prominent and pervasive tools for teacher appraisal and feedback across international contexts, buoyed by the strong relationship between higher student ratings of teachers and additional measures of teacher effectiveness. Yet, there is disagreement and conflicting evidence regarding the underlying structure of student perception survey instruments, including whether students distinguish multiple dimensions of teaching or form more general impressions (or unidimensional conceptions) of teaching. This study examined the structure of the Tripod student survey in Australian schools using a competing model analysis of different survey structures and examination of discriminant validity to identify the best model fit. Findings challenge the purported Tripod model structure and raise larger questions regarding elements of teaching that students distinguish via student perception surveys. Implications for the use of and continued research on student perception surveys, including those originating in other countries, are discussed.

A 360-degree evaluation of the professionalism and communication skills of technologists working in the radiology department of a public sector tertiary care hospital in Karachi, Pakistan

BackgroundRadiologic technologists (RTs) are at the forefront of radiology departments and require optimal skills to demonstrate professionalism and effective provider–patient communication. Hence, the objective of this study was to determine the reliability of 360-degree evaluation as a feedback tool to improve competencies in this crucial workforce.MethodsA planned 360-degree evaluation was conducted using a 10-item tool to evaluate communication skills and professionalism among radiological technologists working at Dow University Hospital, Karachi. In total, 311 evaluations of fifteen radiological technologists were conducted while performing various diagnostic radiology procedures at the radiology department of Dow University Hospital. The data were analysed using SPSS version 26 to compute the evaluation scores and the reliability of 360-degree evaluation.ResultsA total of 311 RT-patient interactions were recorded over 48 days, with scores from all three raters. The evaluation tool was found to have good internal consistency for patients, faculty, and RT, with Cronbach’s alpha values of 0.89, 087, and 0.74, respectively. The study found an intraclass correlation of 0.66 (95% CI; 0.58–0.72), showing moderate reliability of the 360-degree evaluation across different raters.ConclusionWe conclude that 360-degree evaluation is a valid and reliable tool for determining the professionalism and communication skills of radiologic technologists and should be incorporated into training programs for formative and summative assessments. However, large-scale multicenter studies are crucial for generalizing these findings and incorporating 360 evaluations in radiological technologist training programs for formative and summative assessments.

Balancing Honest Assessment and Compassion for Learners Experiencing Burnout: A Workshop and Feedback Tool for Clinical Teachers.

Burnout in medical trainees challenges their work effectiveness and impedes their ability to learn. Teachers in the clinical learning environment (CLE) are well situated to identify burnout and are often responsible for learner assessment. We developed a workshop to improve clinical teachers' identification and understanding of learner burnout while empowering them to provide constructive feedback and support. Building on best-practice feedback techniques and utilizing the Maslach Burnout Inventory (MBI) as a framework, we designed the GetINBurnOUT method to provide feedback and support for learners experiencing burnout. Applying Kern's six-step approach to curriculum development, we created and implemented a workshop for clinical teachers centered on advanced burnout knowledge, application of the MBI to the CLE, and use of the GetINBurnOUT feedback method. Kolb's experiential learning theory informed workshop activities such as group discussion, case practice, and self-reflection. Participants completed surveys immediately after the workshop to assess planned behavior. We delivered the workshop eight times at local, regional, and national faculty development programs/conferences to over 188 participants. Participants rated the workshop favorably, with average scores of 4.5-4.8 out of 5 across all domains and program objectives; all participants planned to make a change to their practice. Positive comments emphasized the topic's importance and the GetINBurnOUT tool's practicality. This workshop can enhance clinical teachers' understanding of burnout and provide them with the tools to address it in the CLE. The GetINBurnOUT method offers a practical approach for providing honest assessments while supporting learners in the CLE.

Identification of cultural conversations in therapy using natural language processing models.

Researchers have historically focused on understanding therapist multicultural competency and orientation through client self-report measures and behavioral coding. While client perceptions of therapist cultural competency and multicultural orientation and behavioral coding are important, reliance on these methods limits therapists receiving systematic, scalable feedback on cultural opportunities within sessions. Prior research demonstrating the feasibility of automatically identifying topics of conversation in psychotherapy suggests that natural language processing (NLP) models could be trained to automatically identify when clients and therapists are talking about cultural concerns and could inform training and provision of rapid feedback to therapists. Utilizing 103,170 labeled talk turns from 188 psychotherapy sessions, we developed NLP models that recognized the discussion of cultural topics in psychotherapy (F-1 = 70.0; Spearman's ρ = 0.78, p < .001). We discuss implications for research and practice and applications for future NLP-based feedback tools. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Automated feedback on fluency and complexity measures for L2 academic paragraphs: Student perspectives and impacts on human-rater scores

While complexity, accuracy, and fluency (CAF) measures are known to correlate with L2 writers’ scores, less is known about the effectiveness of providing automated feedback based on these measures for improving writing performance. Furthermore, it remains unclear if improvements in specific CAF measures correspond to improvements in human-rater scores. Finally, the trade-off hypothesis predicts that all three components of CAF cannot be improved at once, so providing multiple CAF measures to students at the same time might cause cognitive overload and overwhelm students, reducing their ability to uptake the feedback. To examine these issues, a simple paragraph feedback tool was developed to provide input on number of words, vocabulary variety, supporting detail markers, and sentence length. The tool was implemented repeatedly in L2 academic paragraph writing lessons with 124 students. Improvement was assessed through complexity, accuracy, and fluency (CAF) measures and human-rater scores, while also gathering student opinions. Results showed significant improvements in number of words, sentence length, and supporting detail marker usage from pre- to post-treatment with improvements in number of words and vocabulary variety found to be the strongest predictors of improvements in human-rater scores. Students reported finding vocabulary variety feedback most helpful, while supporting detail marker feedback was perceived as most confusing. Notably, students reported very low cognitive load overall. This article also discusses pedagogical implications of these findings for L2 writing instruction and automated feedback implementation.

Description of a Longitudinal Interprofessional Education Plan and Outcomes From an Inaugural Student Cohort

ObjectiveTo describe a longitudinal interprofessional education (IPE) plan at 1 school of pharmacy including inaugural student cohort outcomes. MethodsStandards 2016 prompted the creation of an IPE task force at the University of Wisconsin–Madison School of Pharmacy to develop and implement an IPE plan consisting of outcome-based goals, a deliberate design, and a formalized assessment strategy (an approach that inspired national consensus guidance from the Health Professions Accreditors Collaborative). Beginning with the 2022 graduating class, required interprofessional learning activities were embedded longitudinally across a 4-year Doctor of Pharmacy program and included classroom-, simulation-, and clinical-based IPE activities. Student learning outcomes, focused on attitudes, skills, and behaviors, were assessed longitudinally using 3 self-reported, validated measures allowing before and after comparisons: Student Perceptions of Interprofessional Clinical Education–Revised, Version 2, Interprofessional Education Collaborative Competency Self-Assessment Tool, Version 3, and Interprofessional Collaborative Competencies Attainment Survey–Revised, coupled with the Individual Teamwork Observation and Feedback Tool. ResultsEnd point Student Perceptions of Interprofessional Clinical Education–Revised, Version 2 and Interprofessional Education Collaborative Competency Self-Assessment Tool, Version 3 average scores were in the agree to strongly agree range with statistically significant before and after change in 80% (4 of 5) of subscales. Students’ attainment and demonstration of positive teamwork behaviors were evidenced by statistically significant before and after changes in advanced pharmacy practice experience student Interprofessional Collaborative Competencies Attainment Survey–Revised scores and universal satisfactory ratings of advanced pharmacy practice experience students on the Individual Teamwork Observation and Feedback Tool. ConclusionA developmentally progressive, longitudinal IPE plan was implemented into a 4-year Doctor of Pharmacy program. Inaugural student cohort assessment results documented students were prepared for interprofessional practice upon graduation.

Live recording of how the AI can assess bronchoscopy performance and work as a feedback tool. The bronchoscopists did not receive the AI’s rating or guidance. AI, artificial intelligence.

Live recording of how the AI can assess bronchoscopy performance and work as a feedback tool. The bronchoscopists did not receive the AI’s rating or guidance. AI, artificial intelligence.

Recontextualising the Unified Theory of Acceptance and Use of Technology (UTAUT) Framework to Higher Education Online Marking

The Higher Education (HE) sector has faced an exceptional period of complex and fast-moving change, meaning staff have had to quickly adjust their teaching delivery, reconsider how learning outcomes are assessed, and adopt new technologies. Despite significant efforts by HE staff across these areas, the willingness to adopt new tools and technologies is not consistent across all staff cohorts and indeed some researchers suggest that this willingness, or readiness to adopt emerging technologies and educational tools can be shaped by individual differences, attitudes, and social norms.This study focuses on education technology acceptance in HE via a specific university and explores the use of technology enhanced feedback and marking tools, such as online embedded rubrics. Underpinned by the unified theory of acceptance and use of technology (UTAUT), this research project aims to explore the perceptions of staff towards online feedback and marking practices. In turn, the research project considers the relevancy of the UTAUT model in differing / current contexts. Contribution manifests through a) the methodological choice of qualitative semi-structured interviews to explore the UTAUT dimensions, b) the deep contextual understanding of the challenges HE staff face in adopting and implementing new technologies, c) suggested adaptations to UTAUT, and c) recommendations for HE practice.

A Feedback Tool for the Implementation of Assessment for Learning and Process Writing in English Home Language Writing Lessons

A Feedback Tool for the Implementation of Assessment for Learning and Process Writing in English Home Language Writing Lessons

Exploring the Acceptance of Mobile-Assisted Peer Feedback in English Speaking among Chinese EFL Students

Grounded in the Technology Acceptance Model (TAM), this quasi-experimental study examined the use of Mobile-Assisted Peer Feedback (MAPF) tools via the Tencent Mini Programme among Chinese university EFL learners to enhance English speaking proficiency. A mixed-methods approach was employed, using descriptive statistics for questionnaire data and thematic analysis with NVivo for interview data. Sixty non-English major students (CEFR levels A2-B1) were selected through purposive sampling and assigned to three groups: one-way, dialogic, and visualised peer feedback. Results showed positive responses across all modes, with visualised feedback rated highest for usefulness, behavioural intention, and speaking improvement. The study highlights the need to consider technological, individual, and sociocultural factors in mobile-assisted feedback for language learning. Future research should explore long-term effects, application in diverse contexts, and further refinement of mobile-assisted feedback modes to enhance learner engagement. This study contributes to the research on MAPF in English speaking and offers practical insights for educational technology development.

Examining Writing Feedback Dynamics from ChatGPT AI and Human Educators: a Comparative Study

As technology increasingly influences educational practices, it is crucial to comprehend the impact and effectiveness of AI in comparison to human-driven methodologies. In the field of language learning and writing instruction, one notable development is the emergence of AI-generated writing feedback tools. These tools, exemplified by ChatGPT in this study, promise efficiency, objectivity, and detailed insights for learners. This study aims to investigate the comparative effectiveness of an artificial intelligence (AI) automated writing feedback tool and human teacher feedback in enhancing the writing skills of English Language Learners. The research was conducted on fifteen third-year undergraduate students in the discipline of English Language Education, using a phenomenological design. The study collected qualitative data through assignments and feedback given to students at a higher education institution. The findings suggest that while human teachers were praised for their understanding, personalized guidance, and emotional intelligence in providing feedback, AI-generated feedback was considered more detailed and comprehensive. It highlights the importance of balancing efficiency with the human touch in the feedback proce

The use of ai in learning English: a comparative study between Moroccan and Indonesian undergraduate students from the English department

It is generally understood that people’s attitudes influence and shape their behavior. However, there is still a gap in knowledge regarding how undergraduate students from different countries perceive AI in learning English. This paper aims to compare two groups of undergraduate students from the English departments of Morocco and Indonesia. To achieve this, an online structured questionnaire was administered to 240 undergraduate students from these two countries. Data collected were analyzed using SPSS version 25 and the Mann-Whitney test. The findings revealed that both groups are optimistic about using AI for learning English. They utilize AI tools for feedback, engagement, motivation, personalized learning, and autonomy. However, there is a disparity in how each group uses AI for translation. Additionally, participants expressed concerns about AI related to privacy and security, overreliance and lack of responsibility, critical thinking and creativity issues, and plagiarism. The study suggests that AI can be both beneficial and detrimental. Incorporating AI into the classroom is essential for improving students’ AI literacy and enhancing their academic and language skills. A balanced approach to AI use involves not only providing students with diverse AI tools suited to their academic needs but also raising awareness about the ethical issues associated with AI.

SP1.9 - Management of cutaneous abscesses and infected sebaceous cysts using inhaled methoxyflurane Penthrox®: a 12 month prospective observational study

Abstract Aims Incision and drainage of abscesses contribute a significant burden upon emergency general surgical theatres. The purpose of this study was to look at the safety and effectiveness of incision and drainage using inhaled methoxyflurane (Penthrox®) to reduce reliance upon surgical theatres and provide an effective alternative to local anaesthetic. Methods After a successful pilot of 20 patients, adults with ‘simple’ abscesses were enrolled over a 12-month period and given inhaled methoxyflurane for ward-based incision and drainage. Pilonidal and perianal abscesses, patients with ASA ≥3 and abscesses greater than 5cm in size were excluded. Length of admission and outcomes at 1 week, 1 month and 3 months post procedure were compared to previously audited data on simple abscess drainage in theatre to determine safety, efficacy and cost-savings. Results 130 patients had an abscess drained using Penthrox® inhaled analgesia. 93.8% (122/130) had successful drainage with a mean admission time of 99 minutes. Eight patients reattended for further treatment (3&amp;lt;3 months, 3&amp;lt;1 month and 2&amp;lt;1 week); two patients required antibiotics and five required further drainage. Patients taken to theatre spent on average 34.5 hours in hospital compared to 1.65 hours for those performed using Penthrox®. A patient feedback tool captured very positive comments with no negative experiences recorded. Conclusion Penthrox® is a safe, effective analgesic for the incision and drainage of simple abscesses and patient experience is overwhelmingly positive. In addition, it significantly reduces the burden on emergency theatres and the hospital bed-base with an estimated minimum saving of £1704 per patient.

Developing TOPIK Speaking Practice Model and Exploring Effectiveness Using ChatGPT as a Learning Feedback Tool

Developing TOPIK Speaking Practice Model and Exploring Effectiveness Using ChatGPT as a Learning Feedback Tool

University Students’ Engagement with Feedback in a Quiz Platform: A Case Study of Quizizz

Students’ engagement with feedback is vital for the learning process. It is widely acknowledged that technology can enhance students’ engagement with feedback. However, research on technology-generated feedback tools emphasises that different technologies have varying impacts on students’ engagement. Recently, there has been a dearth of research shedding light on students’ engagement with feedback from quiz platforms. This mixed-methods study aims to examine the extent to which and how students engage with feedback on the Quizizz platform, drawing from Fredrick et al.'s (2004) framework of student engagement, including affective engagement, behavioural engagement, and cognitive engagement. Data were collected from a questionnaire with 294 respondents who were learning English as a foreign language at a private university in Vietnam and three focus group interviews with 15 participants. The findings indicate that participants moderately engaged with feedback from the quiz platform, and these engagements were quite complex. Moreover, data from the interviews revealed aspects of the quiz platform that can enhance students' engagement with feedback. Additionally, an Exploration Factor Analysis was conducted to explore the structure of the questionnaire measuring students’ engagement with feedback, which confirms the usefulness of the framework in measuring students’ engagement with feedback from an interactive platform. The implications for using a quiz platform regarding students’ feedback literacy were discussed.