Unlocking the Potential of Large Language Models in Education: Factors Influencing Adoption by Instructional Designers and Academics

Large language models (LLMs) are widely considered as potential game-changers for English as a foreign language (EFL) teaching and learning, and in particular, there is a growing body of research indicating the transformative role they may play in translation learning. However, little is known about how students actually use and benefit from LLMs in their translation learningand what obstacles they may encounter during their interaction with such tools. To address these gaps, this exploratory case study aimed to examine the affordances of LLMs for translation learning and the constraining factors that may affect learning experiences and outcomes. Thirty Chinese university students volunteered to participate in a 4-week translation project in which they utilized LLMs for translation practice in an out-of-class setting. The analysis revealed the presence of three types of affordances: (1) preparation-related affordances, (2) process-related affordances, and (3) product-related affordances. LLMs significantly enhanced translation learning experience by assisting with preparation, facilitating the process, and refining outputs. The findings also highlighted that LLMs’ assistive capability may be hampered due to either the issues inherent in the technical design of LLMs or learners’ cognitive weakness and insufficient AI literacy. The study holds pedagogical implications concerning the instruction and task design for LLM-aided translation learning.

This study explores and analyzes the specific performance of Large Language Models (LLMs) in instructional design, aiming to unveil their potential strengths and possible weaknesses. Recently, the influence of LLMs has gradually increased in multiple fields, yet exploratory research on their application in education remains relatively scarce. In response to this situation, our research, grounded in Pedagogical Content Knowledge (PCK) theory, initially formulated an instructional design framework based on mathematical problem chains and corresponding prompt instructions. Subsequently, a comprehensive tool for assessing LLM's instructional design capabilities was developed. Utilizing GPT-4, a high school mathematics teaching plan dataset was generated. Finally, the performance of LLMs in instructional design was evaluated. The evaluation results revealed that the teaching plans generated by LLMs excel in setting instructional objectives, identifying teaching priorities, organizing problem chains and teaching activities, articulating subject content, and selecting methods and strategies. Particularly commendable performance was noted in the modules of statistics and functions. However, there is room for improvement in aspects related to mathematical culture and interdisciplinary assessment, as well as in the geometry and algebra modules. Lastly, this study proposes initiatives such as LLM Prompt-based teacher training and the integration of mathematics-focused LLMs. These suggestions aim to advance personalized instructional design and professional development of teachers, offering educators new insights into the in-depth application of LLMs.

Since the emergence of ChatGPT, research on large language models (LLMs) has actively progressed across various fields. LLMs, pre-trained on vast text datasets, have exhibited exceptional abilities in understanding natural language and planning tasks. These abilities of LLMs are promising in robotics. In general, traditional supervised learning-based robot intelligence systems have a significant lack of adaptability to dynamically changing environments. However, LLMs help a robot intelligence system to improve its generalization ability in dynamic and complex real-world environments. Indeed, findings from ongoing robotics studies indicate that LLMs can significantly improve robots’ behavior planning and execution capabilities. Additionally, vision-language models (VLMs), trained on extensive visual and linguistic data for the vision question answering (VQA) problem, excel at integrating computer vision with natural language processing. VLMs can comprehend visual contexts and execute actions through natural language. They also provide descriptions of scenes in natural language. Several studies have explored the enhancement of robot intelligence using multimodal data, including object recognition and description by VLMs, along with the execution of language-driven commands integrated with visual information. This review paper thoroughly investigates how foundation models such as LLMs and VLMs have been employed to boost robot intelligence. For clarity, the research areas are categorized into five topics: reward design in reinforcement learning, low-level control, high-level planning, manipulation, and scene understanding. This review also summarizes studies that show how foundation models, such as the Eureka model for automating reward function design in reinforcement learning, RT-2 for integrating visual data, language, and robot actions in vision-language-action models, and AutoRT for generating feasible tasks and executing robot behavior policies via LLMs, have improved robot intelligence.

In recent years, artificial intelligence technologies represented by large language models (LLMs) have achieved breakthrough progress, driving the development of intelligent applications across various domains. However, LLMs still face several challenges in practical applications, including knowledge update latency, hallucination in generated content, and persistent biases that are difficult to eliminate. This paper investigates the decision-making preferences and behavioral patterns of LLMs when inconsistencies arise between their internal parameterized knowledge and externally retrieved information enhanced by retrieval-augmented techniques. We propose a systematic evaluation method for assessing LLMs’ internal knowledge dependency. First, we establish a data augmentation method for knowledge-intensive question-answering tasks to construct an evaluation dataset. Using the spaCy entity categorization tool combined with an LLM-based verification and filtering mechanism, we classify and annotate entities in QA datasets and corpora. Through LLM-integrated entity recognition and substitution techniques, we generate logically consistent transformed facts and corresponding questions on the TriviaQA dataset, automatically constructing a synthetic transformed fact dataset. Next, we integrate retrieval-augmented LLMs and conduct a systematic evaluation of multiple mainstream LLMs. By comparing model performance differences on original versus transformed facts, we quantitatively analyze their knowledge dependency and generation controllability, revealing significant variations across models and exposing limitations in existing LLM evaluation methods. Additionally, this paper evaluates the impact of different prompting techniques on models’ internal knowledge dependency. Experimental results show that all tested models exhibit varying degrees of reliance on internal knowledge, where chain-of-thought prompting tends to reinforce this dependency, while appropriate instruction design can partially mitigate it. We provide a dialectical analysis of the dual nature of LLMs’ internal knowledge: while such knowledge effectively resists misinformation interference and defends against malicious knowledge injection attacks, it also hinders knowledge updates and reduces model flexibility.

Large Language Models (LLMs) and Generative AI (GenAI) tools have been popularised through the widely publicised launch of tools such as ChatGPT by OpenAI and Copilot by Microsoft. While being able to generate seemingly humanlike language and respond to human input in an apparently conversational manner, these tools are not able to understand problems in the same way that humans do, and their outputs are based on probabilistic algorithms and are susceptible to hallucinations, even with extensive training. As with any tool, LLMs can be used poorly and even unethically, but there are benefits to be gained through comprehension of how LLMs work and how prompt engineering can drastically improve the outputs from LLMs. This paper builds on existing research in the field of prompt engineering while engaging with the ongoing discourse of how AI is impacting higher education and wider society. Utilising Bloom’s revised taxonomy of learning as a foundation, a theoretical model is proposed for the purpose of helping educators analyse and evaluate what types of LLM usage may be appropriate within subject-specific learning experiences. Practical examples of prompts are provided as a starting point for academics and students to use and extend as part of their own experimentation with GenAI tools. The intent of this pedagogy-first approach to the use of emerging AI technologies in education will empower students to develop their higher order thinking skills (HOTS) and help them work towards co-creation in collaboration with LLMs without compromising academic integrity or their own agency.Keywords: Large Language Models; LLMs; Generative AI; GenAI; Artificial Intelligence; Prompt Engineering; Higher Education; Bloom’s TaxonomyPart of the Special Issue Generative AI and education <https://doi.org/10.21428/8c225f6e.0390b853>

Background:In alignment with UNESCO's Sustainable Development Goal 4 (SDG4), which advocates for inclusive and equitable quality education, the integration of Artificial Intelligence tools—particularly Large Language Models (LLMs)—presents promising opportunities for transforming higher education. Despite this potential, empirical research remains scarce regarding the effects of LLM use on students' academic performance, mental well-being, and engagement, especially across different modes of implementation.ObjectiveThis experimental study investigated whether a guided, pedagogically grounded use of LLMs enhances students' academic writing quality, perceived mental health, and academic engagement more effectively than either unguided use or no exposure to LLMs. The study contributes to UNESCO's “Futures of Education” vision by exploring how structured AI use may foster more inclusive and empowering learning environments.MethodA total of 246 undergraduate students were randomly assigned to one of three conditions: guided LLM use, unguided LLM use, or a control group with no LLM access. Participants completed a critical writing task and standardized instruments measuring academic engagement and mental well-being. Prior academic achievement was controlled for, and writing quality was assessed using Grammarly for Education.ResultsStudents in the guided LLM condition achieved significantly higher scores in writing quality and academic engagement compared to the control group, with large and moderate effect sizes, respectively. Modest improvements in mental health indicators were also observed. By contrast, unguided use yielded moderate gains in writing quality but did not produce significant effects on engagement or well-being.ConclusionThe findings highlight the critical role of intentional instructional design in the educational integration of AI tools. Structured guidance not only optimizes academic outcomes but also supports students' wellbeing and inclusion. This study offers empirical evidence to inform ongoing debates on how digital innovation can contribute to reducing educational disparities and advancing equitable learning in the post-pandemic era.

Recent rapid developments in generative artificial intelligence (genAI), including large language models (LLMs), have the potential to radically transform higher education—perhaps especially the work of learning designers. This paper investigates how learning designers are making sense of and adapting to the current challenges and future changes caused or enabled by ongoing advances in genAI and LLMs. A team of learning designers was surveyed to gain insights into how they are currently engaging with genAI and LLMs in their professional practice and in teaching and learning contexts. This paper describes how these learning designers see genAI and LLMs emerging as not only useful tools but as agential actors and collaborators in teaching and learning contexts, and how they see this changing their practice and profession now and potentially into the future.

Large language models (LLMs), which are a specific subset of artificial intelligence (AI), may have the potential to revolutionize education by addressing common student misconceptions in physics. This study investigates the effectiveness of popular LLMs, such as OpenAI ChatGPT, Google Gemini, and Microsoft Copilot, in identifying and addressing misconceptions related to Newtonian mechanics in high school physics. The focus solely is on the misconception that for an elevator to move upward, the force exerted by the cable must be greater than the gravitational force acting downward on the elevator, which contradicts Newton’s first law of motion. To explore this, thirty-two experienced instructors engaged in dialogues with the LLMs, simulating learners with this misconception. Instructors evaluated the LLMs’ accuracy, personalization, and pedagogical effectiveness. The findings indicate that most instructors recognized the substantial potential of LLMs to improve student learning, particularly in addressing misconceptions through interactive dialogue, targeted questioning, and clear explanations tailored to each learner’s needs. ChatGPT ranked highest, demonstrating capabilities in delivering clear explanations, adapting to individual learners, and implementing effective teaching strategies. Google Gemini followed closely, while Microsoft Copilot was the least effective. This capability holds promise for enhancing conceptual understanding and student engagement in physics education. However, limitations were noted in the LLMs’ ability to facilitate personalized scientific discussions and utilize visual aids, such as physics diagrams, simulations, to enhance understanding. This research demonstrates the significant potential of LLMs as valuable tools for identifying and addressing misconceptions in physics education.

This study explores the impact of robot–LLM (Large Language Model) integration on collaborative creative writing, focusing on how embodiment and AI creativity influence various aspects of creative output. A total of 150 undergraduate students participated in a structured experimental design with five collaboration conditions: Human–Human (HH), Human–Computer with High‐Creativity LLM (HC), Human–Robot with High‐Creativity LLM (HR), Human–Robot with Low‐Creativity LLM (RL) and Human–Computer with Low‐Creativity LLM (CL). Creativity was assessed through expert ratings and computational analysis of originality, imagery, voice and semantic flow. The results revealed that while the Human–Robot (High‐Creativity LLM) condition significantly enhanced originality, Human–Human and Human–LLM (text‐based) collaborations excelled in imagery and voice. The study identified an ‘embodiment paradox’, where robot embodiment amplified creativity in high‐creativity AI conditions, yet human collaboration remained superior in stylistic expression. Mediation analysis revealed that user engagement acted as a mediator, with embodiment compensating for low‐creativity AI and amplifying the creative process with high‐creativity AI. The findings have important implications for the design of collaborative AI systems, highlighting the need for a balanced integration of embodiment and AI creativity to optimize creative outcomes. This research contributes to our understanding of how human–robot–LLM collaborations can expand creative potential in writing, offering insights for future AI applications in educational and creative industries. Practitioner notes What is already known about this topic? Previous studies have explored the impact of AI in creative collaborations, with a focus on text‐based models like LLMs enhancing writing quality. Embodiment in AI systems, such as humanoid robots, has been shown to affect user engagement and emotional responses, influencing creativity. Human collaboration has traditionally been seen as superior in generating stylistic elements like imagery and voice, while AI excels in originality and idea generation. What this paper adds? This research demonstrates that robot–LLM collaboration significantly boosts originality, particularly when high‐creativity AI is used. The study uncovers the ‘embodiment paradox’, where embodied robots enhance creativity in high‐creativity AI conditions but human collaboration remains superior in stylistic expression. The mediation role of user engagement is explored, showing how embodiment can enhance creative outcomes when AI creativity is low and amplify them when AI creativity is high. Implications for practice and/or policy Educators and trainers can utilize embodied AI systems in creative tasks to increase student or participant engagement and foster more original outputs. Training programmes can be structured to leverage the strengths of both human collaboration and AI, tailoring tasks based on AI's creativity levels for optimal outcomes. Policy around the integration of AI in educational and creative settings should encourage balanced AI systems that combine embodiment and creativity for enhanced collaborative work.

IntroductionLarge language model has seen rapid uptake in education alongside advances in artificial intelligence. Its capabilities in areas such as healthcare planning and question answering suggest strong potential for supporting personalised instruction in physical education. At the same time, studies have raised concerns regarding safety, accuracy, and contextual appropriateness. This study examines the role of large language model in physical education and evaluates its suitability as an assistive tool or possible alternative to human instruction.MethodsThis study conducted a questionnaire—based evaluation comparing three generative approaches for producing physical education lesson plans and teacher responses: (1) plans written directly by a large language model, (2) plans produced by a large language model after being provided with domain resources, and (3) plans created collaboratively by physical education teachers together with the large language model. Feasibility, practicality, safety, adaptability, and content quality were assessed across these approaches. Large language model—generated responses in typical physical education scenarios were further evaluated in terms of response accuracy, safety, clarity, adaptability, guidance quality, acceptability to learners, and perceived potential to enhance teacher response efficiency.ResultsLesson plans produced collaboratively by physical education teachers and the large language model outperformed those produced solely by the large language model—both with and without additional resources—across all evaluated dimensions. In addition, responses generated by the large language model in physical education scenarios were rated highly for clarity, guidance, adaptability, and support for teacher efficiency.DiscussionThe large language model demonstrates clear value in physical education, particularly as a means to support and augment instructional design and responsiveness. However, it does not yet replace the pedagogical judgment, contextual awareness, and safety oversight of human instructors. Overall, this study concludes that the large language model is best positioned as an assistive tool to optimise teaching practice rather than as a standalone replacement for human teachers.

Large language model (LLM) chatbots have rapidly emerged as powerful tools in education, offering new avenues to support social and emotional learning (SEL). Literature lacks synthesis studies related to SEL competencies and how LLM chatbots are used to foster them, that is, affordances. Therefore, this systematic mapping review categorizes and synthesizes current literature on the utilization of LLM chatbots for SEL development in higher education. Following PRISMA, we screened and analysed 43 peer‐reviewed studies, identifying 15 competencies across six SEL domains (cognitive, emotional, social, values, perspectives and identity), and 19 LLM chatbot affordances across 7 genres. Findings show that cognitive competencies (e.g., critical thinking, metacognition) and identity competencies (e.g., self‐efficacy, agency) are most frequently targeted, whereas values and perspective domains remain underrepresented. The most common affordances were immediate personalized feedback, information retrieval and interactive question and answer. We also addressed five ethical risks (transparency, privacy, equality, beneficence and affect/identity safety). We proposed an SEL development framework for higher education students using LLM chatbots. By focusing on these specific competencies and affordances, researchers can contribute to the advancement of SEL through the innovative use of LLM chatbots. We encourage researchers to conduct more studies to expand this framework by adding, removing or revising the competencies and affordances. Context and implications Rationale for this study: While large language model (LLM) chatbots are increasingly used in education, no comprehensive review existed to synthesize how their specific affordances map into the development of distinct social and emotional learning (SEL) competencies. A review is needed to address this critical gap for designing effective, ethically aware educational interventions in education. Why the new findings matter: This review provides the first structured framework linking 19 LLM affordances to 15 SEL competencies. The framework reveals a significant imbalance in development focus (e.g., cognitive overvalues) and a widespread lack of essential ethical safeguards, which is crucial for maximizing benefits and minimizing risks. Implications: For researchers, it provides a foundational SEL–LLM framework and agenda to explore underrepresented domains (e.g., values). Practitioners (educators, instructional designers) can use the affordance‐competency map to design targeted SEL interventions. Policy‐makers should prioritize ethical guidelines and support the development of specialized, equitable LLM tools for SEL.

Objective: The objective of this study is to explore the potential of integrating Large language models (LLMs) into higher education to enhance teaching effectiveness. It investigates how LLMs can support personalized learning, improve teacher - student interaction, and foster content innovation. The study also addresses the challenges associated with the use of LLMs, including the transformation of the teacherʼs role, data privacy concerns, and technical limitations in handling complex cognitive tasks. Methods: A mixed - method approach was used, combining a literature review, survey, and case study analysis. The literature review focused on artificial intelligence applications in education, while a survey was conducted among 120 professors and 533 students from five universities in China to gather quantitative data on their experiences with AI in education. Case studies were also analyzed to assess the effectiveness of LLM - supported learning platforms in enhancing classroom engagement, interaction, and teaching outcomes. Results: The survey results revealed that 68% of professors and 74% of students found LLMs beneficial for personalized learning and improved classroom engagement. However, 58% of professors raised concerns regarding the changing role of teachers and data privacy issues, while 49% of students worried about over - reliance on AI affecting their independent learning. Case studies showed a 30% improvement in teacher - student interaction and a 25% increase in student engagement, although LLMs struggled with advanced cognitive tasks in specialized fields such as mathematics. Conclusion: The study concludes that while LLMs offer significant advantages in improving personalized learning and enhancing interaction, their integration into higher education must be managed carefully. Teacher training, ethical considerations, and data privacy safeguards are essential. Future research should focus on optimizing LLMs for specialized academic fields and exploring their combination with emerging technologies like virtual reality and augmented reality to create more interactive learning environments.

Self‐directed learning (SDL) is a critical skill in the 21st century, particularly in online Python learning environments. Learning analytics (LA) can track and analyse learning processes, which can be leveraged to prompt students to reflect on their learning strategies and progress through learning analytics dashboards (LADs). However, LADs lack pedagogical domain knowledge and fail to provide effective personalised feedback and guidance. This study designs and presents a Generative AI‐powered SDL tool, SDLChat. It integrates a large language model (ERNIE‐3.5) with retrieval‐augmented generation (RAG) technology to generate contextualised, actionable feedback for learners across the entire SDL cycle: planning, self‐monitoring and self‐reflection. To evaluate the impact of SDLChat on learners' SDL skills and Python knowledge, a randomised experimental study was conducted over a six‐week Python online course. The study compared the changes in SDL skills and Python knowledge of students using both SDLChat and LAD group ( n = 39) and LAD‐only group ( n = 35). The results indicate that: (1) students using SDLChat and LAD significantly outperformed those using LAD alone in Python knowledge mastery, self‐monitoring and interpersonal skills and (2) the LAD‐only group showed significant improvement only in Python knowledge mastery; however, (3) no significant differences were found in posttask motivation between these two groups. This study highlights the potential of integrating LLM with learning analytics to enhance SDL skills and learning performance in online learning contexts. It also establishes a theory‐informed operational framework for understanding the LLM‐empowered SDL process. Practitioner notes What is already known about this topic Self‐directed learning (SDL) is essential for success in online learning environments, requiring learners to plan, manage, monitor and reflect on their learning processes. Learning analytics (LA), particularly in the form of learning analytics dashboards (LADs), is commonly used to track SDL processes and encourage learner reflection. Traditional LADs are incapable of providing personalised feedback, limiting their effectiveness in enhancing SDL skills and learning performance. What this paper adds Introduces SDLChat, an LLM‐powered SDL tool combining a large language model (ERNIE‐3.5) and retrieval‐augmented generation (RAG) technology to generate contextualised and actionable feedback across the full SDL cycle. Provides empirical evidence from a quasi‐experimental study demonstrating that the integration of SDLChat and a LAD enhances self‐monitoring and interpersonal skills. Highlights the superiority of the integration of SDLChat and LAD in improving learning performance. Proposes an AI4SDL operational framework by including a technological dimension to extend SDL theory in online learning environments. Implications for practice and/or policy Educators and instructional designers can leverage AI‐powered tools like SDLChat to provide personalised feedback, fostering key SDL skills and improving learning outcomes in online environments. Policymakers should establish SDL skills as curricular objectives and implement professional development programmes to enhance teachers' digital literacy and their capacity for human–AI collaborative instruction. Institutions offering online courses may benefit from adopting AI‐driven solutions to enhance student engagement, self‐monitoring and academic performance, potentially improving course completion rates and learner satisfaction.

The accessibility of advanced Artificial Intelligence-based tools, like ChatGPT, has made Large Language Models (LLMs) readily available to students. These LLMs can generate original written content to assist students in their academic assessments. With the rapid adoption of LLMs, exemplified by the popularity of OpenAI's ChatGPT, there is a growing need to explore their application in education. Few studies examine students' use of LLMs as learning tools. This paper focuses on the application of ChatGPT in engineering higher education through an in-depth case study. It investigates whether engineering students can generate high-quality university essays with LLMs assistance, whether existing LLMs identification systems can detect essays produced with LLMs, and how students perceive the usefulness and acceptance of LLMs in learning. The research adopts a deductive/inductive approach, combining conceptualization and empirical evidence analysis. The study involves mechanical and management engineering students, who compose essays using LLMs. The essay assessment showed good results, but some recommendations emerged for teachers and students. Thirteen LLMs detectors were tested without achieving satisfactory results, suggesting to avoid LLMs ban. In addition, students were administered a questionnaire based on constructs and items that follow the technology acceptance models available in the literature. The results contribute to qualitative evidence by highlighting possible future research and educational practices.

Integration of Large Language Models (LLMs) with Learning Management Systems (LMSs) can enhance task automation and accessibility in education. However, hallucination where LLMs generate inaccurate or misleading information remains a challenge. This study introduces the Dynamic Course Content Integration (DCCI) mechanism, which dynamically retrieves course content from Canvas LMS and structures it within an LLM’s context window via prompt engineering, enabling the LLM-powered assistant, Ask ME, to deliver context-aware, curriculum-aligned responses while mitigating hallucinations. A mixed-methods pilot study grounded in Self-Determination Theory (autonomy, competence) and the Technology Acceptance Model (perceived usefulness, ease of use) evaluated DCCI’s effectiveness with 120 first-year programming students at Eötvös Loránd University. The course focused on foundational programming patterns in C#, including writing program specifications. We analyzed 14,746 logged interactions and a post-course survey completed by 101 students. User satisfaction was measured via a 5-point Likert scale (turn-level ratings), while the survey assessed usability, engagement, and ethical concerns. Results indicated high satisfaction (mean 4.65/5) and strong recognition of Ask ME’s ability to provide timely, contextually relevant answers to administrative and course-related queries. 78.06% agreed that Ask ME’s Canvas integration reduced platform switching, improving usability, engagement, comprehension, and topic exploration. Many students reported reduced hesitation to ask questions and increased motivation for self-directed learning, though concerns about over-reliance on AI and reduced student–teacher interaction emerged. This study demonstrates that DCCI enhances LLM reliability, student satisfaction, and engagement in AI-driven educational automation, while highlighting the importance of balancing AI efficiency with pedagogical integrity and human–AI collaboration.