Text Feedback Research Articles

Enhancing online learning: a systematic literature review exploring the impact of screencast feedback on student learning outcomes

Purpose This systematic literature review paper critically examines the effectiveness of screencast feedback compared with text feedback in promoting student learning outcomes in online higher education. This paper aims to contribute to the ongoing discussion surrounding feedback modalities and their impact on online learning environments.Design/methodology/approach This paper adopts a systematic review approach to synthesise and analyse existing studies investigating the use of screencast feedback in online higher education settings. A comprehensive search and selection process was employed to identify relevant literature. The selected studies were then analysed for their methodologies, findings and implications. This paper seeks to provide an overview of the current state of research, highlighting the benefits, challenges and potential impacts of screencast feedback on student learning outcomes.Findings The findings of this paper suggest that while there is a positive perception of screencast feedback among students and instructors, drawing definitive conclusions about its superiority over text feedback remains at the very beginning. Students generally appreciate the personalised, supportive and engaging nature of screencast feedback, particularly within the online learning context. However, challenges such as technical barriers and potential workload implications for instructors are also noted. Further empirical research is needed to comprehensively evaluate the comparative efficacy of screencast feedback, considering factors like online engagement, digital literacy and the impact on diverse student populations.Research limitations/implications This review underscores the acute necessity for expansive and meticulously designed studies that can provide conclusive insights into the authentic potential of screencast feedback and its resonance within the unique landscape of online learning. Through rigorous inquiry, educators can discern the optimal strategies for harnessing the advantages of screencast feedback to enhance student learning outcomes, aligning harmoniously with the dynamics of virtual classrooms.Practical implications Screencast feedback emerges as a promising avenue to foster meaningful connections between instructors and learners. The review highlights that screencast feedback engenders a more dialogic interaction between lecturers and students, resulting in personalised, supportive and engaging feedback experiences.Social implications The systematic review conducted underscores the positive reception of screencast feedback from both students and lecturers in this context. The findings are consistent with the principles of social constructivist theory, suggesting that the interactive and personalised nature of screencast feedback facilitates a richer educational experience for students, even within the confines of virtual classrooms (Vygotsky, 1978).Originality/value This innovative blend of methodologies contributes new insights that can inform educational practices and pedagogical strategies in online learning environments.

Unveiling the efficacy of ChatGPT in evaluating critical thinking skills through peer feedback analysis: Leveraging existing classification criteria

This study investigates the potential of using ChatGPT, a large language model, to assess students' critical thinking in online peer feedback. With the rapid development of technology, big language models, such as ChatGPT, have made significant progress in natural language processing in recent years and have good potential for application in teaching evaluation and feedback. However, can generative AI help educational practitioners in teaching and learning? How to accurately assess students' critical thinking using generative AI remains a challenging task. This study investigates whether ChatGPT can effectively evaluate critical thinking using established coding systems. By comparing the consistency and accuracy of manual coding with ChatGPT coding in online peer feedback texts, it clarifies how ChatGPT processes online peer feedback data and conducts assessments. Through a comprehensive analysis employing various metrics including precision, recall, F1 score, and a confusion matrix, we assess ChatGPT's performance. Additionally, we group students and analyze how ChatGPT's assessments relate to their critical thinking levels. Our findings suggest that the ChatGPT demonstrated some ability to assess higher dimensions of critical thinking, but showed limitations in assessing the more granular secondary dimensions under the higher dimensions of critical thinking. However for this kind of granular assessment will more accurately capture the level of learning critical thinking. Surprisingly, ChatGPT's evaluations are not influenced by students' critical thinking levels. This study underscores ChatGPT's potential in automating critical thinking assessment at scale, alleviating the burden on educators and enhancing understanding of critical thinking in peer feedback.

How well do faculty do in providing general surgery EPA feedback?

IntroductionEntrustable Professional Activities (EPAs) provide a framework for competency-based assessment in surgery. EPA descriptions include observable behaviors by trainees at progressive levels of autonomy. The American Board of Surgery (ABS) required all General Surgery (GS) residency programs to implement assessment of 18 EPAs at the beginning of academic year 2023–2024. Microassessments provide formative self-reflection by the resident and feedback by faculty upon completion of the EPA. These frequent assessments culminate in a resident performance profile utilized by the trainee for formative growth and the clinical competency committee for summative feedback. Assessor free text comments are an opportunity to provide meaningful, constructive feedback to residents. Our aim was to analyze comments provided by faculty to residents in terms of their alignment with EPA descriptors and provision of actionable feedback. MethodsA total of 540 ​GS EPA assessments for inguinal hernia, gallbladder disease, appendicitis, trauma, and surgical consultation were evaluated from 6/2021-12/2022. We assessed free text EPA comments from faculty compared to EPA behavior descriptions for alignment with the selected EPA level of entrustment. The comments were judged on a binary scale of “Align” vs “Not Align” by two independent evaluators, with a third evaluator to address discordance. Comments were then evaluated for resident behavioral descriptions, suggestions for improvement, and positive or negative feedback. ResultsApproximately 77 ​% of EPA microassessments had alignment between level of autonomy and free text feedback. A common example of feedback discordant with level of autonomy was rating a trainee at an intraoperative level 4 (independent practice) with comments such as “required some guidance with retrocecal case and upsizing port.” Based on behavior descriptions this would be a level 3 (indirect supervision). Approximately 88 ​% of feedback contained positive comments with minimal negative feedback (e.g., “this did not go well.”). Actionable feedback including “work on optimization of retracting hand” or “continue to work clamp/tie technique and square off each knot” was present in 28.3 ​% of feedback. ConclusionsThe majority of faculty provide feedback that is aligned with the behavioral anchors of the EPAs assessed, but frequently did not provide actionable feedback to the resident regarding how to advance to the next level of entrustment. EPA entrustment behaviors provide a framework for the development of practice-ready behaviors, and if assessors anchor their feedback in the behaviors for a given entrustment level and project how a resident could proceed to the next level, they can provide a clear trajectory for skill development. Faculty development should focus on improving the frequency of actionable free text feedback, outlining how residents can advance in the future.

Harnessing large language models to auto-evaluate the student project reports

Addressing the problem of the difficulty in providing timely and reasonable feedback evaluation for student project reports, this paper proposes a method based on LLMs (Large Language Models) that can automatically generate instant feedback evaluations for student project reports. Three LLMs, namely BART (Bidirectional and Auto-Regressive Transformer), CPTB (chatgpt_paraphraser_on_T5_base), and CGP-BLCS (chatgpt-gpt4-prompts-bart-large-cnn-samsum), were designed to generate instant text feedback pre-training models for student project reports. The effectiveness of the feedback was evaluated using ROUGE Metrics, BERT Scores, and human expert evaluations. Experiments showed that the lightweight, fine-tuned BART model, when trained on a larger dataset of 80%, generated effective feedback evaluations for student project reports. When trained on a smaller dataset of 20%, both the BART and CPTB models had unsatisfactory overall performance, while the fine-tuned CGP-BLCS model was able to generate feedback evaluations that approached human-level evaluations. The detailed descriptions of the methods used with the LLMs for generating effective text feedback evaluations for student project reports will be useful to AI computer programmers, researchers, and computer science instructional designers for improving their courses and future research.

Leveraging text mining and analytic hierarchy process for the automatic evaluation of online courses

This study introduced a multi-criteria decision-making methodology leveraging text mining and analytic hierarchy process (AHP) for online course quality evaluation based on students’ feedback texts. First, a hierarchical structure of online course evaluation criteria was formulated by integrating topics (sub-criteria) identified through topic modeling and interpreted based on transactional distance and technology acceptance theories. Second, the weights of the criteria in the hierarchical structure were determined based on topic proportions. Third, the AHP was employed to determine the overall relative advantage of online courses and their relative advantage within each criterion based on the hierarchical framework and criterion weights. The proposed approach was implemented on the datasets of 6940 reviews for knowledge-seeking courses in Art, Design, and Humanities (D1) and 44,697 reviews for skill-seeking courses in Computer Science, Engineering, and Programming (D2) from Class Central to determine ranking positions of nine courses from both D1 and D2 as alternatives. Results revealed common concerns among knowledge and skill-seeking course learners, encompassing “assessment”, “content”, “effort”, “usefulness”, “enjoyment”, “faculty”, “interaction”, and “structure”. The article provides valuable insights into the online course evaluation and selection processes for learners in D1 and D2 groups. Notably, both groups prioritize “effort” and “faculty”, while D2 learners value “assessment” and “enjoyment”, and D1 learners value “usefulness” more. This study demonstrates the efficacy of leveraging online learner reviews and topic modeling for automating MOOC evaluation and informing learners’ decision-making processes.

Exploring student interpreters’ engagement with different sources of feedback on note-taking

ABSTRACT Note-taking is a critical part of interpreting teaching and learning. During note-taking training, feedback is crucial to help student interpreters hone relevant skills, and the effectiveness of feedback hinges on students’ engagement. This case study investigates students’ affective, cognitive, and behavioural engagement with feedback on notes from two major sources: peer and teacher. Eight student interpreters were recruited and divided into two groups. The feedback texts, interpreting notes, study journals, and interview transcriptions were coded and analysed. While the two groups both had positive and negative feelings, they preferred teacher feedback. Students’ cognitive engagement showed differences in meta-cognitive operations. Some students’ behavioural engagement was inconsistent with the other two dimensions. The results suggest that students should utilise different sources of feedback to achieve their learning goals, and the study provides some insights for teachers to improve feedback activities.

Enhancing E-Learning Adaptability with Automated Learning Style Identification and Sentiment Analysis: A Hybrid Deep Learning Approach for Smart Education

In smart education, adaptive e-learning systems personalize the educational process by tailoring it to individual learning styles. Traditionally, identifying these styles relies on learners completing surveys and questionnaires, which can be tedious and may not reflect their true preferences. Additionally, this approach assumes that learning styles are fixed, leading to a cold-start problem when automatically identifying styles based on e-learning platform behaviors. To address these challenges, we propose a novel approach that annotates unlabeled student feedback using multi-layer topic modeling and implements the Felder–Silverman Learning Style Model (FSLSM) to identify learning styles automatically. Our method involves learners answering four FSLSM-based questions upon logging into the e-learning platform and providing personal information like age, gender, and cognitive characteristics, which are weighted using fuzzy logic. We then analyze learners’ behaviors and activities using web usage mining techniques, classifying their learning sequences into specific styles with an advanced deep learning model. Additionally, we analyze textual feedback using latent Dirichlet allocation (LDA) for sentiment analysis to enhance the learning experience further. The experimental results demonstrate that our approach outperforms existing models in accurately detecting learning styles and improves the overall quality of personalized content delivery.

Analysis of Textual Feedback of Students for Course Evaluation in Universities Through Machine Learning Algorithms

Many educational institutions worldwide make significant efforts to collect student feedback to understand their perspectives on the courses and faculty. This feedback is used to enhance the institution's environment. In this modern world, institutions use data collection techniques to gather feedback. However, they lack the proper techniques to analyze and utilize this data to improve the educational quality of the institute using textual feedback. This study presents techniques for analyzing the written feedback from students, which was collected for course evaluation over a year. This paper focuses on techniques including Multinomial Naive Bayes Classifier, Long Short-Term Memory(LSTM), and Random Forest to enhance the outcomes of sentiment analysis. Ultimately, our efforts resulted in the LSTM achieving 97.45% accuracy during model testing for three types of sentiments: positive, neutral, and negative. This paper also aims to identify a clear research gap in this field and discusses the work of other researchers, including their less accurate models from the past. We also discuss the processes of collecting a sufficient amount of data to train this model, and then utilize a set of 25,689 data points for training. Furthermore, this paper primarily focuses on enhancing the quality of education. The initial model has been implemented at Balochistan UET Khuzdar, and it has produced satisfactory results. In the future, efforts will be made to find the perfect way to enhance the quality of education.

Effectiveness of an Artificial Intelligence-Assisted App for Improving Eating Behaviors: Mixed Methods Evaluation.

A plethora of weight management apps are available, but many individuals, especially those living with overweight and obesity, still struggle to achieve adequate weight loss. An emerging area in weight management is the support for one's self-regulation over momentary eating impulses. This study aims to examine the feasibility and effectiveness of a novel artificial intelligence-assisted weight management app in improving eating behaviors in a Southeast Asian cohort. A single-group pretest-posttest study was conducted. Participants completed the 1-week run-in period of a 12-week app-based weight management program called the Eating Trigger-Response Inhibition Program (eTRIP). This self-monitoring system was built upon 3 main components, namely, (1) chatbot-based check-ins on eating lapse triggers, (2) food-based computer vision image recognition (system built based on local food items), and (3) automated time-based nudges and meal stopwatch. At every mealtime, participants were prompted to take a picture of their food items, which were identified by a computer vision image recognition technology, thereby triggering a set of chatbot-initiated questions on eating triggers such as who the users were eating with. Paired 2-sided t tests were used to compare the differences in the psychobehavioral constructs before and after the 7-day program, including overeating habits, snacking habits, consideration of future consequences, self-regulation of eating behaviors, anxiety, depression, and physical activity. Qualitative feedback were analyzed by content analysis according to 4 steps, namely, decontextualization, recontextualization, categorization, and compilation. The mean age, self-reported BMI, and waist circumference of the participants were 31.25 (SD 9.98) years, 28.86 (SD 7.02) kg/m2, and 92.60 (SD 18.24) cm, respectively. There were significant improvements in all the 7 psychobehavioral constructs, except for anxiety. After adjusting for multiple comparisons, statistically significant improvements were found for overeating habits (mean -0.32, SD 1.16; P<.001), snacking habits (mean -0.22, SD 1.12; P<.002), self-regulation of eating behavior (mean 0.08, SD 0.49; P=.007), depression (mean -0.12, SD 0.74; P=.007), and physical activity (mean 1288.60, SD 3055.20 metabolic equivalent task-min/day; P<.001). Forty-one participants reported skipping at least 1 meal (ie, breakfast, lunch, or dinner), summing to 578 (67.1%) of the 862 meals skipped. Of the 230 participants, 80 (34.8%) provided textual feedback that indicated satisfactory user experience with eTRIP. Four themes emerged, namely, (1) becoming more mindful of self-monitoring, (2) personalized reminders with prompts and chatbot, (3) food logging with image recognition, and (4) engaging with a simple, easy, and appealing user interface. The attrition rate was 8.4% (21/251). eTRIP is a feasible and effective weight management program to be tested in a larger population for its effectiveness and sustainability as a personalized weight management program for people with overweight and obesity. ClinicalTrials.gov NCT04833803; https://classic.clinicaltrials.gov/ct2/show/NCT04833803.

Customer Feedback Analysis Using Text Mining

Complexity surrounding the holistic nature of customer experience has made measuring customer perceptions of interactive service experiences challenging. At the same time, advances in technology and changes in methods for collecting explicit customer feedback are generating increasing volumes of unstructured textual data, making it difficult for managers to analyze and interpret this information. Consequently, text mining, a method enabling automatic extraction of information from textual data, is gaining in popularity. However, this method has performed below expectations in terms of depth of analysis of customer experience feedback and accuracy. In this study, we advance linguistics-based text mining modeling to inform the process of developing an improved framework. The proposed framework incorporates important elements of customer experience, service methodologies and theories such as co-creation processes, interactions and context. This more holistic approach for analyzing feedback facilitates a deeper analysis of customer feedback experiences, by encompassing three value creation elements: activities, resources, and context (ARC). Empirical results show that the ARC framework facilitates the development of a text mining model for analysis of customer textual feedback that enables companies to assess the impact of interactive service processes on customer experiences. The proposed text mining model shows high accuracy levels and provides flexibility through training. As such, it can evolve to account for changing contexts over time and be deployed across different (service) business domains; we term it an “open learning” model. The ability to timely assess customer experience feedback represents a pre-requisite for successful co-creation processes in a service environment.

Use of Topic Analysis for Enhancing Healthcare Technologies

Nowadays, technology has played a vital role in the advancement of the healthcare sector. Various healthcare datasets are available on the web in the form of patents, scientific papers, articles, textual feedback, chatlogs, abstracts of papers, medical reports, and social media posts. It is a tedious task for the stakeholders to find hidden crucial knowledge on the discussed topic from this content, which if utilized optimally can lead to the rapid development of the healthcare sector. Topic analysis concepts are very effective in extracting meaningful topics from the data. Here, frequently applied Topic modeling methods -Latent Semantic Analysis, Probabilistic Latent Semantic Analysis, Latent Dirichlet Allocation, Correlated Topics Model, and Non-negative matrix factorization are surveyed along with their benefits and drawbacks. Insights on new innovative topic modeling techniques used in healthcare with their objective, opportunities, and challenges are provided, which can help the researchers for the enhancement of healthcare facilities.

Alerta Cannabis: A Tailored-Computer Web-Based Program for the Prevention of Cannabis Use in Adolescents: A Cluster-Randomized Controlled Trial Protocol

BackgroundThe growing use of cannabis in adolescence is a public health problem that must be addressed through prevention. In Spain, the average age of initiation of cannabis use in the adolescent population is 14.8 years. At 14 years, the lifetime prevalence of cannabis use is 11.7%, which increases to 51.,5% at the age of 18; the prevalence of cannabis use in the population aged 14 to 18 years is 28.6%, a figure that must be tried to reduce, that is why this school prevention program is proposed: Alerta Cannabis.MethodsThe Alerta Cannabis research project consists of design, implementation, and evaluation. In the first phase, a computer-tailored eHealth program (Alerta Cannabis) is developed based on the I-Change Model, an integrated model based on three main behavioral change processes: awareness, motivation, and action. This program consists of four 30-minute sessions that will provide culturally adapted and personalized advice to motivate students not to use cannabis through text feedback, animations, and gamification techniques. This phase will also include usability testing. In the implementation phase, secondary school students from Western Andalusia, Spain (Seville, Cádiz, Huelva, and Córdoba) and Eastern Andalusia (Jaén, Málaga, and Granada) will be randomized to an experimental condition (EC) or a control condition (CC) for a cluster randomized clinical trial (CRCT). Each condition will have 35 classes within 8 schools. GI will receive the online intervention Alerta Cannabis. EC and CC will have to fill out a questionnaire at baseline, six months, and twelve months of follow-up. In the last phase, the effect of Alerta Cannabis is evaluated. The primary outcomes are the lifetime prevalence of cannabis use and its use in the last 30 days and at 6 months. At 12 months of follow-up, the prevalence in the last 12 months will also be assessed. The secondary outcome is the intention to use cannabis.DiscussionThe study tests the effect of the innovative program specifically aimed to reduce the use of cannabis in the adolescent population through eHealth in Spain. The findings aim to develop and implement evidence-based cannabis prevention interventions, which could support school prevention, for instance, the assistance of school nurses. If the program proves to be effective, it could be useful to prevent cannabis use on a national and international scale.Trial registrationNCT05849636. Date of registration: March 16, 2023.

“Thoroughly out of my depth - A quality improvement project to improve junior clinician’s confidence in undertaking adult ADHD clinic appointments

IntroductionThere has been a 1000% increase in referrals for assessment of adult ADHD within Scotland over the past three years (The Scottish Government. 2023. NAIT adult Neurodevelopmental Pathways Report). These referrals are sent by general practitioners to the local community mental health team. The most junior clinicians (doctors who are pre-membership with the Royal College of Psychiatrists) in the team are often responsible for undertaking the initial assessment of these patients. Patients have on average waited almost a year to be seen and expectations are high.ObjectivesThe diagnosis of ADHD can be challenging, and adult ADHD is still a relatively new and evolving diagnostic entity. We set out to explore how junior clinicians were coping with this in their daily practice.MethodsWe developed a questionnaire that was sent to all junior clinicians working within Argyll &amp; Bute (n=8) via an anonymised email link. The link was open for 1 week and then results were analysed.ResultsThe response rate to our survey was 87.5%. Prior to starting their current roles none of the respondents had ever undertaken an ADHD assessment before. All respondents answered “No” when asked if they felt they had adequate knowledge on ADHD in order to perform assessments. Only 14% (n=1) felt they had access to adequate resources about how to make an ADHD diagnosis. Participants were asked on a scale of 1-10 (1= not at all,10= very) to rate their confidence in conducting ADHD assessments. The average confidence score was 2.43. There was a space for free text feedback in which participants reported the following: “felt thoroughly out my depth”, “I felt chucked in the deep end”, “I felt very under qualified”Image:ConclusionsOur results showed that junior clinicians did not feel confident or knowledgeable about undertaking adult ADHD assessments. In response to this we have now produced, in association with the consultant psychiatrists within our health board, an informative and engaging quick reference poster which explains how to undertake an ADHD assessment (see QR code attached as Image 1). It contains useful pointers about diagnosis, and more importantly guidance on language/phrases to use when explaining to patients whether or not you feel a diagnosis of ADHD is appropriate. It then also explains next steps that can be offered i.e. psychological support/medication options and how to prescribe these. We plan to send out a repeat questionnaire to the next cohort of junior clinicians to assess if they have improved confidence in managing and diagnosing ADHD following this intervention.Disclosure of InterestNone Declared

Prioritising national healthcare service issues from free text feedback – A computational text analysis & predictive modelling approach

Patient experience surveys have become a key source of evidence for supporting decision-making and continuous quality improvement within healthcare services. To harness free-text feedback collected as part of these surveys for additional insights, text analytics methods are increasingly employed when the data collected is not amenable to traditional qualitative analysis due to volume. However, while text analytics techniques offer good predictive capabilities, they have limited explanatory features often required in formal decision-making contexts, such as programme monitoring or evaluation. To overcome these limitations, this study integrates computational text and predictive modelling as part of a Computational Grounded Theory method to determine the effect of quality gaps in care dimensions and their prioritisation from free-text feedback. The feedback was collected as part of a national survey to support decisions on continuous improvement in Maternity Services in Ireland. Our approach enables (1) operationalising the service quality lexicon in the context of maternity care to explain the effect of quality gaps in care dimensions on overall satisfaction from free-text comments; and (2) extending the service quality lexicon with two organisational and political decision-making concepts: “Salience” and “Valence”, for prioritising perceived quality gaps. These methodological affordances enable the extension of service quality theory to explicitly support the prioritisation of improvement decisions which before now required additional decision frameworks. Results show that tangibles-, process-, and reliability-related care issues have the highest importance in our study context. We also find that hospital contexts partly determine the relative importance of gaps in care dimensions.

Exploring an effective automated grading model with reliability detection for large‐scale online peer assessment

AbstractBackgroundPeer assessment has played an important role in large‐scale online learning, as it helps promote the effectiveness of learners' online learning. However, with the emergence of numerical grades and textual feedback generated by peers, it is necessary to detect the reliability of the large amount of peer assessment data, and then develop an effective automated grading model to analyse the data and predict learners' learning results.ObjectivesThe present study aimed to propose an automated grading model with reliability detection.MethodsA total of 109,327 instances of peer assessment from a large‐scale teacher online learning program were tested in the experiments. The reliability detection approach included three steps: recurrent convolutional neural networks (RCNN) was used to detect grade consistency, bidirectional encoder representations from transformers (BERT) was used to detect text originality, and long short‐term memory (LSTM) was used to detect grade‐text consistency. Furthermore, the automated grading was designed with the BERT‐RCNN model.Results and ConclusionsThe effectiveness of the automated grading model with reliability detection was shown. For reliability detection, RCNN performed best in detecting grade consistency with an accuracy rate of 0.889, BERT performed best in detecting text originality with an improvement of 4.47% compared to the benchmark model, and LSTM performed best with an accuracy rate of 0.883. Moreover, the automated grading model with reliability detection achieved good performance, with an accuracy rate of 0.89. Compared to the absence of reliability detection, it increased by 12.1%.ImplicationsThe results strongly suggest that the automated grading model with reliability detection for large‐scale peer assessment is effective, with the following implications: (1) The introduction of reliability detection is necessary to help filter out low reliability data in peer assessment, thus promoting effective automated grading results. (2) This solution could assist assessors in adjusting the exclusion threshold of peer assessment reliability, providing a controllable automated grading tool to reducing manual workload with high quality. (3) This solution could shift educational institutions from labour‐intensive grading procedures to a more efficient educational assessment pattern, allowing for more investment in supporting instructors and learners to improve the quality of peer feedback.

SPIRIT: Style-guided Patch Interaction for Fashion Image Retrieval with Text Feedback

Fashion image retrieval with text feedback aims to find the target image according to the reference image and the modification from the user. This is a challenging task, as it requires not only the synergistic understanding of both visual and textual modalities but also the ability to model a wide variety of styles that fashion images contain. Hence, the crucial aspect of addressing this problem lies in exploiting the abundant semantic information inherent in fashion images and correlating it with the textual description of style. Recognizing that style is generally situated at the local level, we explicitly define style as the commonalities and differences between local areas of fashion images. Building upon this, we propose a Style-guided Patch InteRaction approach for fashion Image retrieval with Text feedback (SPIRIT), which focuses on the decisive influence of local details of fashion images on their style. Three corresponding networks are designed pertinently. The Patch-level Style Commonality network is introduced to fully leverage the semantic information among patches and compute their average as the style commonality. Subsequently, the Patch-level Style Difference network employs a graph reasoning network to model the patch-level difference and filter out insignificant patches. By considering the above two networks, mutual information about style is obtained from the interaction between patches. Finally, the Visual Textual Fusion network is utilized to integrate visual features with rich semantic information and textual features. Experimental results on four benchmark datasets demonstrate that our proposed SPIRIT achieves state-of-the-art performance. Source code is available at https://github.com/PKU-ICST-MIPL/SPIRIT_TOMM2024 .

Abstract B040: Former caregivers of patients with glioblastoma multiforme report the Preparedness Assessment for the Transition Home (PATH©) instrument is helpful throughout the disease trajectory

Abstract Purpose: The purpose of the study was to determine if the Preparedness Assessment for the Transition Home (PATH©), a validated caregiver assessment instrument, would be helpful in identifying gaps in preparation of primary caregivers of patients with glioblastoma multiforme (GBM). The PATH© was designed to assess caregiver readiness to assist the healthcare team in developing tailored solutions to meet the needs of caregiver/patient dyads. Methods: Former primary caregivers (spouses or life-partners) of now deceased patients with GBM who were members of the private Facebook (FB) group “Widows and Widowers of GBM” reviewed the instrument and completed a web-based survey via Qualtrics. Following IRB approval (#H22-0103) and approval from the FB group administrator, a survey link was posted on the FB group page and left open for 3 weeks in Spring 2023. Participants were asked to review the 25 PATH© items and respond to 4 yes/no or Likert scale questions about whether the instrument would have helped to better prepare them for the caregiving role across the phases of GBM disease trajectory (acute, living with disease progression, end-of-life). A free text option was included so caregivers could provide suggestions for tool revision to better address the needs of caregivers of patients with GBM. Summary statistics were calculated for interval and ratio variables. ANOVA and t-test were conducted to determine whether there were differences across stages among the variables (stage of disease trajectory and helpfulness score). Chi-square and Fisher’s exact test were conducted to examine whether helpfulness and overall helpfulness were independent. A linear regression was conducted to assess whether the different phases of the disease trajectory predicted tool helpfulness. The GBM Caregiver Study team met to review, analyze, code free text feedback, and identify and define code themes. Results: Demographics were completed by 191 study participants; 124 reviewed the PATH© instrument and completed the survey. Ninety-two percent of participants reported that the PATH© instrument would be helpful across all stages of the disease trajectory, with the majority selecting “Very” or “Somewhat” helpful. Only 8% of participants did not think the instrument would be helpful at end-of-life, and 6% of participants did not think the instrument would be helpful at other stages of the disease trajectory. Fifty-four participants suggested changes to the existing instrument for GBM caregivers. From these open-ended responses, four themes were identified: education, access to care team and resources, care at the end-of-life/after death, and self-care. Conclusion: Study results indicated using the PATH© instrument in the GBM population would be helpful to assess caregiver preparation. Study collaborators are actively planning future research, building on the findings of this study, to explore the impact of how implementing a systematic caregiver assessment and tailored care plans and/or interventions can improve caregiver readiness and reduce caregiver burden. Citation Format: Elizabeth James, Marlee Wallace, Megan Chard, Barbara Lutz, Laurie Minns. Former caregivers of patients with glioblastoma multiforme report the Preparedness Assessment for the Transition Home (PATH©) instrument is helpful throughout the disease trajectory [abstract]. In: Proceedings of the AACR Special Conference on Brain Cancer; 2023 Oct 19-22; Minneapolis, Minnesota. Philadelphia (PA): AACR; Cancer Res 2024;84(5 Suppl_1):Abstract nr B040.

Promoting second language writing through technology-driven multimodal text feedback

ABSTRACT Purpose Despite the increasing use of technology-driven multimodal pedagogies in second-language writing education, there is a dearth of studies investigating the value and design of multimodal text feedback. Based on Chong’s (Chong, S. W. 2019. “College Students’ Perception of e-Feedback: A Grounded Theory Perspective.” Assessment & Evaluation in Higher Education 44 (7): 1090–1105. https://doi.org/10.1080/02602938.2019.1572067) tripartite conceptualization of written feedback, this article examines the effectiveness of the multimodal text feedback approach by exploring its influence on the technical, socio-emotional, and personal aspects of feedback. Design/methodology/approach The study was conducted in a second-tier university in southeastern China. The researchers collected 57 participants’ responses to a questionnaire survey regarding the effectiveness of multimodal text feedback and then conducted semi-structured interviews with 17 participants. Findings Qualitative and quantitative analysis confirms multiple positive impacts of multimodal text feedback, such as improving technical features of students’ writing and enhancing teacher-student relationships. Originality/value The findings not only inform language-writing instructors with regard to the process-based design of multimodal text feedback but also lead to the proposal of a comprehensive conceptual framework for evaluating the interplay between different aspects of feedback.

MetRoBERTa: Leveraging Traditional Customer Relationship Management Data to Develop a Transit-Topic-Aware Language Model

Transit riders’ feedback provided in ridership surveys, customer relationship management (CRM) channels, and, in more recent times, through social media, is key for transit agencies to better gauge the efficacy of their services and initiatives. Getting a holistic understanding of riders’ experience through the feedback shared in those instruments is often challenging, mostly because of the open-ended, unstructured nature of text feedback. In this paper, we propose leveraging traditional transit CRM feedback to develop and deploy a transit-topic-aware large language model (LLM) capable of classifying open-ended text feedback to relevant transit-specific topics. First, we utilize semi-supervised learning to engineer a training dataset of 11 broad transit topics detected in a corpus of 6 years of customer feedback provided to the Washington Metropolitan Area Transit Authority. We then use this dataset to train and thoroughly evaluate a language model based on the RoBERTa architecture. We compare our LLM—MetRoBERTa—with classical machine learning approaches utilizing keyword-based and lexicon representations. Our model outperforms those methods across all evaluation metrics, providing an average topic classification accuracy of 90%. Finally, we provide a value proposition of this work demonstrating how the language model, alongside additional text processing tools, can be applied to add structure to open-ended text sources of feedback such as Twitter. The framework and results we present provide a pathway for an automated, generalizable approach for ingesting, visualizing, and reporting transit riders’ feedback at scale, enabling agencies to better understand and improve customer experience.

The advantage of videos over text to boost adolescents' lateral reading in a digital workshop

ABSTRACT Today, fostering media and information literacy (MIL) among citizens is essential for preserving democracy. The effectiveness of scalable interventions, particularly those employing lateral reading, a fact-checking heuristic, is well recognised. Our digital workshop, designed to model lateral reading from a user's perspective, has proven beneficial in educational settings and has garnered expert endorsement. Addressing recent scholarly calls to understand the underlying mechanisms of successful MIL interventions, our study is anchored in Cognitive Load Theory. We examine how different instructional modalities in our workshop influence engagement with lateral reading and how effectively it is executed. The study separates implicit text feedback and hands-on video instructions, previously combined, in a full-factorial design using a parallel-group RCT. We analysed the responses of 178 upper-secondary students in the online workshop and conducted a detailed video analysis of 30 students participating in a controlled environment at their schools. Our findings reveal that hands-on video instructions notably enhance both the engagement and effectiveness of the lateral reading heuristic. This study underscores the significance of a human-computer interaction perspective in designing more impactful media and information literacy interventions.