Evaluation Of Learning Outcomes Research Articles

Interprofessional Team Training With Virtual Reality: Acceptance, Learning Outcome, and Feasibility Evaluation Study.

Effective interprofessional teamwork is vital for ensuring high-quality patient care, especially in emergency medicine. However, interprofessional education often fails to facilitate meaningful interaction among health care disciplines. It is therefore imperative to afford early opportunities for cultivating interprofessional teamwork skills. While in-person simulation-based training has been shown to improve performance, this is resource-intensive, especially if it involves multiple professions. Virtual reality (VR)-based training is an innovative instructional approach that demands fewer resources and offers the flexibility of location-independent learning. This study aimed to develop and evaluate the acceptance, learning outcome, and feasibility of an interprofessional team (INTEAM) training course that included a VR simulation of a neurological emergency case. This 1-group study used a pre- and posttest design to evaluate the 2-hour INTEAM training course for nursing and medical students. The course included an e-learning part, VR simulation, and debriefing. The main learning objectives were derived from the entrustable professional activity 6, namely to handle a common problem in emergency medicine (headache due to subarachnoid hemorrhage and epileptic seizure) that requires interprofessional collaboration, including a structured handover. We used validated and self-constructed questionnaires, pre- and posttests, and open questions to assess the acceptance, learning outcome, and feasibility of the course. The data of 42 students (21 nursing and 21 medical students) were analyzed and showed good usability in the System Usability Scale (median 72.5, IQR 65-80). The perception of usefulness (median 6, IQR 5.8-6.9) and ease of use (median 5.9, IQR 5.1-6.3) was good among all students. There was a significant increase in the handover performance from pre- (median 8, IQR 6-9) to posttraining (median 8, IQR 7-9; z=-2.01; P=.045; r=0.33) and of the confidence in caring for patients with seizures (median 3, IQR 2-3 and median 3.5, IQR 3-4, respectively; z=-3.8; P<.001; r=0.60). In 67% (14/21) of the simulations, technical issues occurred, but all simulations could be carried out completely. The new INTEAM training course was well received by nursing and medical students. The handover skills and confidence in caring for patients with seizures were improved after the course. Despite technical challenges with the VR simulations, none required termination, and this demonstrates that our approach is feasible. These promising results encourage the use of VR simulations for team training in the education of nursing and medical students.

Implementation of an Integrated Problem Based Learning Model in Generic Science Skills in Thermochemical Material

This research aims to determine the effect of Prior Knowledge Evaluation, Student Worksheets on student learning outcomes using the Problem Based Learning Model Integrated Generic Science Skills on Thermochemical Material. This research was carried out at SMAN 2 Percut Sei Tuan in the 2022/2023 academic year, the research was designed using a dual paradigm with two independent variables, namely Prior Knowledge Evaluation and Student Worksheets and one dependent variable, namely Evaluation of Learning Outcomes. The population in this study was all class XI at SMAN 2 Percut Sei Tuan. The sample used class XI B as an experimental class taught with Prior Knowledge Evaluation, Student Worksheets and Evaluation of Learning Outcomes tools using the Generic Science Skills Integrated PBL Learning Model. Sampling was using purposive sampling technique. The instruments used in this research consisted of 25 Prior Knowledge Evaluation questions, 15 Student Worksheets questions and 28 Evaluation of Learning Outcomes in multiple choice form which had been developed by previous researchers. The results of this study that thereare significant influence between Prior Knowledge Evaluation and Student Worksheets on learning outcomes. There are differences in learning outcomes between students who have low Prior Knowledge Evaluation and high Prior Knowledge Evaluation.

Implementation of an Integrated Project Based Learning Model for Generic Science Skills in Acid-Base Material

This research aims to determine the effect of Evaluation of Prior Knowledge and Student Worksheets on Evaluation of Learning Outcomes and determine the difference in learning outcomes between high initial knowledge and low initial knowledge using the Project Based Learning model integrated with Generic Science Skills in Acid-Base material. This research has two independent variables, namely Initial Knowledge Evaluation and Learner Worksheet and one dependent variable, namely Evaluation of learning outcomes data. The population in this study were all class XI students of SMA Negeri 2 Kabanjahe consisting of 6 classes. The sample in this study was taken using a purposive sampling technique from 1 class, namely class. The instruments used are initial knowledge evaluation test instruments, learning outcome evaluation test instruments and Student Worksheets as well as non-test student observation sheets. Data were analyzed using multiple linear regression tests and Independent Sample T-Test. The results of data analysis show that there is a significant influence between the value of Prior Knowledge and Student Worksheets on learning outcomes and there is a significant difference in the value of learning outcomes between students whose initial abilities are classified as high and students whose initial abilities are classified as low.

Reframing Education: Evaluating the Transformative Effects of the 2013 Curriculum

This research aims to analyze the impact of implementing the 2013 Curriculum, focusing on consistent implementation, evaluation of learning outcomes, acceptance by students and parents, and adaptation of teaching methods and continuous feedback. The research method involves direct observation and interviews with educators, students, parents, and school administrators to understand curriculum implementation and its impact comprehensively. The findings show that consistent curriculum implementation produces a structured learning experience while evaluating learning outcomes. Increases student academic performance. The positive reception from students and parents and adapting innovative teaching methods enrich the learning process. A system of feedback and continuous improvement ensures the relevance and effectiveness of the curriculum. The conclusion of this research confirms that implementing the 2013 Curriculum effectively supports the achievement of educational goals by improving the quality of learning and stakeholder involvement.

Curriculum Management of Local Content in Fostering Religious Behavior: A Study at Madrasah Aliyah

This study investigates the management of local content curriculum in shaping religious behavior at MA Raden Paku Wringinanom. Specifically, the research addresses three key questions: (1) How is the local content curriculum planned to foster religious behavior at MA Raden Paku Wringinanom? (2) How is the local content curriculum implemented to shape religious behavior at MA Raden Paku Wringinanom? (3) What are the outcomes of the local content curriculum in fostering religious behavior at MA Raden Paku Wringinanom? The objectives of this study are to: (1) examine the planning of the local content curriculum, (2) explore the implementation process, and (3) analyze the outcomes in shaping religious behavior at MA Raden Paku Wringinanom. This research employs a qualitative method with descriptive analysis, utilizing an interactive model. The data analysis involves stages of data reduction, data display, and conclusion drawing. After collecting and categorizing the data, the researcher provides an in-depth description before concluding the findings based on available sources. The results of this study are as follows: (1) The planning of the local content curriculum involves adherence to the School Work Plan (RKT) and the Madrasah Work Plan and Budget (RKAM) to define local content programs, select subjects, assign teachers, and determine learning resources and funding. (2) The implementation phase includes syllabus review, lesson plan (RPP) development, and assessment. (3) The outcomes of the local content curriculum include program-specific local content initiatives and the evaluation of student learning outcomes.

3D-printed model is a useful addition in orthopedic resident education for the understanding of tibial plateau fractures

This study aimed to explore the role of the three-dimension (3D) printed models in orthopedic resident training of tibial plateau fractures. A total of 41 residents from our institution were divided into two groups. The intervention group, consisting of 20 residents, had access to 3D-printed models illustrating thirteen tibial plateau fractures. In contrast, the control group, comprising 21 residents, received digital images of thirteen identical tibial plateau fractures. Evaluation of learning outcomes included the accurate identification of tibial plateau fracture patterns, deduction of traumatic mechanisms, preoperative plan, assessment time, and subjective questionnaire responses. The participants with 3D printed models scored significantly higher in both the Schatzker classification and Luo three-column classification compared to those without 3D printed models. Residents in the intervention group performed better in accuracy in deducing traumatic mechanisms compared to the control group. In addition, the sum score of preoperative plan in the intervention group was significantly higher than that in the control group. Specifically, participants with 3D printed models scored higher in surgical approach choice and implants placement than these in the control group. Residents exposed to 3D printed models also spent less time to complete the assessment than those with access only to digital imaging. Subjective assessments indicated that 3D-printed models boosted confidence in fracture identification, improved preoperative plan for fracture management and enhanced the understanding in injury mechanism of tibial plateau fractures. Furthermore, residents agreed that the use of 3D-printed models heightened their interest in learning tibial plateau fractures. Therefore, the addition of 3D printed models significantly contributed to a comprehensive understanding of tibial plateau fractures, the improvement in fracture identification, inferring injury mechanisms and preoperative plan.

Applying Cognitive Linguistics to elucidate the meanings of the particles IN/OUT and UP/DOWN in L2 classrooms

Abstract Cognitive Linguistics offers valuable insights for second language instruction, particularly in enhancing motivated polysemy to facilitate vocabulary acquisition. This paper reports on a longitudinal, classroom-based study and analyses the design, implementation, and assessment of activities inspired by Cognitive Linguistics, elucidating the polysemous meanings of the particles IN/OUT and UP/DOWN. The study involved an experimental group comprising 81 Spanish secondary school students and a control group of 26 students. Following a pre-test/post-test design, participants completed two tests (a gap-fill particle test and a lexical depth test) designed to measure students’ command of polysemous meanings. Despite positive teacher feedback and a perceived increase in awareness of polysemy, the experimental learners showed no statistically significant improvement in the test results. We conclude that a comprehensive approach is required to evaluate learning outcomes, encompassing pedagogical experience, classroom-based research factors, and effective assessment measures.

Thematic control and criteria-based assessment of foreign language writing skills using artificial intelligence technologies

Importance. Currently, there is a tendency that when teaching a foreign language, teachers are increasingly resorting to the use of artificial intelligence technologies to plan training sessions, generate educational content, as well as to conduct automated testing of formed communication skills. The monitoring of academic achievements and assessment is one of the key components of the organization of the educational process. Traditional methods of control and assessment require significant time and labor-intensive costs from the teacher, while artificial intelligence technologies make it possible to simplify and automate routine tasks: check tests and written papers, analyze them and identify mistakes, provide feedback. Thanks to the integration of natural language processing (NLP) technologies into chatbots and adaptive intelligent learning systems, it becomes possible on a daily basis to check texts created by students, evaluate them from the point of view of grammatical and lexical correctness, as well as identify stylistic and factual errors in real time. The purpose of this work is to test the applicability of artificial intelligence technologies for thematic control and criteria-based assessment of educational achievements using the example of productive and reproductive writing.Research Methods. In carrying out this study, the following groups of methods were used: theoretical methods aimed at familiarizing with scientific and methodological literature on the topic of research, analysis and classification of theoretical and methodological material for conducting thematic control and criterion assessment in foreign language classes, as well as empirical methods that allowed modeling pedagogical control and assessment processes using artificial intelligence technologies, observation, analysis and description of the results obtained.Definition of Concepts. The main concepts used in the study are “control of educational results” and its variety “thematic control”, “criterion assessment”.Results and Discussion. In the course of the study, various types of monitoring educational results using artificial intelligence technologies were considered: preliminary, current, intermediate, thematic, final. The choice of thematic control is due to the opportunity to trace its applicability within the framework of a single lesson in a foreign language and identify the main difficulties in using this form of control. The following criteria for evaluating writing skills are used to evaluate learning outcomes using artificial intelligence technologies: a) the structure of the written text; b) compliance with the main topic; c) coherence; d) relevance; e) grammatical correctness; f) lexical correctness; g) ethics of writing and stylistic correctness.Conclusion. Artificial intelligence technologies at the present stage have a high degree of adaptability and include a wide range of software and hardware solutions that allow for such important pedagogical procedures as monitoring educational achievements and evaluation in accordance with user-defined evaluation criteria. The obtained results are proposed to be used in research devoted to the study of modern methods of monitoring educational achievements in the methodology of teaching foreign languages using artificial intelligence technologies.

Immersive Virtual Reality (IVR) Systems for Comprehensive Fire Safety Education in Primary School

In the evolving landscape of educational technology, immersive virtual reality (IVR) has emerged as a promising tool to enhance safety education for primary school students. Traditional fire safety methods, while foundational, face challenges in engaging young learners actively. This study aims to comprehensively investigate the intricate dynamics of integrating IVR technology into fire safety education for primary school students. By examining students' perceptions, assessing engagement levels, and evaluating learning outcomes, the research seeks to offer nuanced insights into the potential benefits and challenges associated with VR-based safety education. Conducted as a qualitative study, 25 semi-structured interviews were performed with primary school students (Grades 4-6 in China), employing a three-step thematic analysis. The research explores students' experiences with IVR fire safety simulations, comparing them with traditional methods, and addressing practical considerations for integration. Preliminary findings suggest positive perceptions, heightened engagement, and improved learning outcomes among students exposed to VR simulations. Practical challenges, including logistical considerations, were identified. The comparison with traditional methods illuminated strengths and weaknesses, guiding educators and policymakers. This research contributes to the intersection of education and technology by offering novel insights into the potential of IVR simulations in fire safety education. The study's significance lies in informing pedagogical practices, addressing practical challenges, and promoting safety awareness among primary school students, guiding future educational technology initiatives.

A study of project-based learning to intermediate EFL learners in reading class: enhancing self-regulated learning of post-secondary students in Macao

AbstractThis study explored two new pedagogical conceptions—project-based learning (PBL) and self-regulated learning (SRL), to enhance effective English teaching and learning in Macao, and to prepare EFL learners for life-long learning. PBL makes English instruction engaging and SRL empowers learners to take ownership of their learning process. The combination of these two methodologies can facilitate students to develop into independent and efficient English learners. The study, guided by Zimmerman’s cyclical phase model of SRL and English and Kitsantas’ PBL/SRL model, was action research which aimed to investigate whether PBL could positively influence the development of intermediate EFL learners’ SRL ability at the post-secondary level in Macao. Research instruments included questionnaires, interviews, students’ reflective notes and teacher’s field notes. Both qualitative and quantitative findings indicated improvement in the students’ SRL ability, reflected in three aspects—motivation, metacognition, and behaviors. The students’ motivational beliefs were significantly strengthened, and more self-regulated behaviors were generated. Though the use of metacognitive strategies did not showcase as significant enhancement as the former elements through the study, the experimental group made progress in using some specific strategies, such as evaluating learning outcomes and making study plans. The summary of the study is that PBL was beneficial to the development of SRL. This research endeavor not only contributes to the existing body of knowledge but also serves as a catalyst for educators to advocate policy reforms, fostering the design of learner-centered, flexible, and coherent post-secondary EFL curricula.

Immersive Virtual Reality (IVR) Systems for Comprehensive Fire Safety Education in Primary School

In the evolving landscape of educational technology, immersive virtual reality (IVR) has emerged as a promising tool to enhance safety education for primary school students. Traditional fire safety methods, while foundational, face challenges in engaging young learners actively. This study aims to comprehensively investigate the intricate dynamics of integrating IVR technology into fire safety education for primary school students. By examining students' perceptions, assessing engagement levels, and evaluating learning outcomes, the research seeks to offer nuanced insights into the potential benefits and challenges associated with VR-based safety education. Conducted as a qualitative study, 25 semi-structured interviews were performed with primary school students (Grades 4-6 in China), employing a three-step thematic analysis. The research explores students' experiences with IVR fire safety simulations, comparing them with traditional methods, and addressing practical considerations for integration. Preliminary findings suggest positive perceptions, heightened engagement, and improved learning outcomes among students exposed to VR simulations. Practical challenges, including logistical considerations, were identified. The comparison with traditional methods illuminated strengths and weaknesses, guiding educators and policymakers. This research contributes to the intersection of education and technology by offering novel insights into the potential of IVR simulations in fire safety education. The study's significance lies in informing pedagogical practices, addressing practical challenges, and promoting safety awareness among primary school students, guiding future educational technology initiatives.

Implementing Learner Autonomy in English as a Foreign Language Classroom Through a Flipped Classroom Pedagogical Approach

This study is a small-scale action research on the problem of using the Flipped Classroom pedagogical approach to enhance learner autonomy in the target group of high school students in an EFL classroom. The main stages of the research were conducted in the spring semester of the academic year of 2023/24. The participants of the study are seven Ukrainian Grade 9 students and three teachers. The research aims at designing and implementing a method for engaging learners in autonomous practices through the Flipped Classroom model. The methodological basis of this study is the works of (Benson &amp; Voller, 2014), the Flipped Learning Network (2014), (Moore, 2015), (Velegol, Zappe, &amp; Mahoney, 2015), (O’Flaherty &amp; Phillips, 2015), (Hung, 2015), (Huang &amp; Hong, 2016), (Nichols, Burgh, &amp; Kennedy, 2017), (Olakanmi, 2017), (Låg &amp; Sæle, 2019), (Jung, Park, Kim, &amp; Park, 2022), (Little, 2022) and other scientists. In this research, the mixed-method approach is used; the quantitative and qualitative data have been collected through guided observation, surveys (teacher interviews and student questionnaires), and mathematical statistics (descriptive and frequency methods). The essence of the designed method lies in the gradual engagement of students in reviewing short video lectures and reading relevant input before class; then, in class, discussing content and doing quizzes and exercises to reinforce understanding, and, finally, actively participating in creating content-related presentations or projects. The results of this study demonstrate that the Flipped Classroom approach has a considerable impact on learner self-awareness, beliefs, and attitudes implemented in autonomous classroom practices, in particular: effective use of free time for English studying; increased motivation to explore new content and understanding it; perception of a teacher as a facilitator, suggesting a move towards a more student-centered or collaborative learning environment; perception of knowledge as something to be gained rather than “transmitted”, taking on responsibility for evaluating learning outcomes through self-assessment or collaborative evaluation methods. Nevertheless, there are some limitations to using FC in the English language classroom: the willingness of teachers to apply learner-centered methods; the willingness of students to take responsibility for their learning; learners’ cognitive and metacognitive skills; the difficulty of the learning material meant for flipping (self-study), access to resources and technology. The results of this study can be used by researchers and practitioners working in the sectors of secondary and tertiary education.

Quantitative evaluation of problem-based learning outcomes on student pharmacists’ patient care process in Japan

Quantitative evaluation of problem-based learning outcomes on student pharmacists’ patient care process in Japan

Design and practice of human-machine cooperative international Chinese character teaching in ChatGPT application

The incorporation of artificial intelligence (AI) into language education has created new opportunities for improving the instruction and acquisition of Chinese characters. Nevertheless, the cognitive difficulties linked to the acquisition of Chinese characters, such as their intricate visual features and lack of clear meaning, necessitate thoughtful deliberation when developing AI-supported learning interventions. The objective of this project is to explore the capacity of a collaborative method between humans and machines in teaching Chinese characters, utilising the advantages of both human expertise and AI technology. We specifically investigate the utilisation of ChatGPT, a substantial language model, for the creation of instructional materials and evaluation methods aimed at teaching Chinese characters to individuals who are not native speakers. The study utilises a mixed-methods approach, which involves both qualitative examination of lesson plans created by ChatGPT and quantitative evaluation of student learning outcomes. The results indicate that the suggested framework for human-machine collaboration can successfully tackle the cognitive difficulties associated with learning Chinese characters, resulting in enhanced learner involvement and performance. Nevertheless, the research also emphasises the constraints of AI-generated material and the significance of human involvement in guaranteeing the accuracy and dependability of educational interventions. This research adds to the expanding collection of literature on AI-assisted language learning and offers practical insights for educators and instructional designers who aim to use AI tools into Chinese language curriculum. The results emphasise the necessity of employing a multi-disciplinary strategy in AI-supported language learning, incorporating knowledge from cognitive psychology, educational technology, and second language acquisition.

A Review of the ECO415 Course Learning Outcome (CLO) Based on the UiTM Melaka Exit Entrance Survey (EES)

The evaluation of program learning outcomes for continuous quality improvement and assessment has garnered global interest in higher academic institution. This is because, student evaluations have the potential to improve the quality of education and the results of learning in a particular course. This article examines the Course Learning Outcomes (CLO) of the Economics course for degree students in UiTM Melaka. This research utilizes data from 328 students who registered ECO415 course for Semester March - August 2023. To evaluate the achievement, Exit Entrance Survey used to compare student responses towards similar questions in the beginning and end of the semester. The question was form in a Likert scale to gauge achievement gaps. Findings demonstrated strong achievement in all three CLOs, with over 70% of students performing at the "excellent" or "very good".

MAIN STAGES OF THE LANGUAGE TEST DEVELOPMENT PROCESS AND VALIDATION ANALYSIS METHODS: THE CASE OF GAZI TÖMER

Testing is the most widely used assessment tool in measuring the development process of students' languagecompetence. Therefore, the existence of valid and reliable language tests appropriate to student achievements in education is of great importance. Research on language test development in foreign language teaching in Kazakhstan is insufficient. The aim ofthis study is to practically demonstrate the basic stages of the test development process and how to conduct content validity analysis with the example of the proficiency test of foreign language (Turkish) prepared by Gazi TÖMER. The research method was based on a quantitative research design. The sample of the research for validation was consisted of 7 experts with experience in Gazi TÖMER. For reliability, there were a total of 66 students studying at Abai KazNPU. The first part of the study was examinedfor the content validity of the relevant test. For this purpose, based on the criteria determined by Webb for the evaluation of learning outcomes, the opinions received from field experts were evaluated using the Lawshe technique. The CVR value of the 45-question test was .96, and the CVI was “significant”. The second part of the study was about reliability. For this purpose, the difficulty level of each item was examined based on the data obtained from the pilot application of the test. Data were analyzedusing Excel and SPSS programs. Additionally, the KR-20 value was calculated and found to be .80, and the test was considered reliable. It is believed that the steps and results included in the study will guide the process of creating valid and reliable tests in various fields.

Aswaja Material Learning Assistance with Peer Learning Method

This study discusses the assistance of learning Aswaja material through the application of peer learning methods in the religious education environment at the madrasah tsnawiyah level. The peer learning method was chosen as an approach to increase students' understanding and involvement in learning Aswaja concepts. This mentoring process involves students as facilitators in study groups, where they share knowledge and learning experiences with each other. A qualitative approach was used in this study, with case studies as the main framework. The research participants consisted of students, teachers, and school administrators who were directly involved in the learning process. Data was collected through interviews, participatory observations, as well as analysis of curriculum documents and evaluation of student learning outcomes. The results of the study show that the use of the peer learning method in Aswaja learning is able to increase student engagement, strengthen understanding of the material, and encourage collaboration between students. Challenges faced in the application of this method include differences in abilities between students and the need for facilitator training to be more effective in guiding groups. The discussion highlights the implications of these findings for the development of the Aswaja learning method in religious education institutions as well as strategic recommendations to optimize the application of peer learning methods to improve the quality of education.

Development and implementation of digital pedagogical support systems in the context of educational equity

In response to the covid-19 pandemic, the digital strategy of education has become a key pedagogical approach for higher engineering disciplines. In this complex environment, the outcome-oriented higher engineering discipline faces the challenge of educational equity in terms of curriculum design and learning outcome assessment for its online engineering practical courses. In order to solve this problem, this study develops a digital teaching assistant system to facilitate the reform of educational equity in higher engineering disciplines. The teaching and learning functions of the system aim to support the interaction between teachers and students in the teaching process and the visual display of learning outcomes, and its intelligent evaluation and analysis function is based on the multi-criterion group decision-making model of Quality Function Deployment (QFD) combined with t-test to realize the evaluation of course learning outcomes. The results show that the academic performance of this system is significantly better than that of other similar systems. The statistical results of the teacher-student satisfaction survey of the two systems show that the teacher-student interaction and multi-dimensional learning evaluation analysis report function provided by the system are the most highly recognized, and the system also has better educational equity in resource recommendation, interaction and collaboration, learning achievement display and learning achievement evaluation, etc. In addition, the system is more in line with the teaching characteristics of higher engineering disciplines, which can provide effective assistance for teachers of higher engineering disciplines to improve teaching plan and cultivate students’ disciplinary practicality, consequently creating favorable conditions for the establishment of teaching equity.

Pathology students' perceptions of virtual learning: A case study of students in Saudi Arabia.

Pathology laboratory classes are traditionally conducted using a conventional light microscope. The Coronavirus Disease 2019 (COVID-19) pandemic and recent technological advances necessitated remote learning through online classes using virtual slides (VS) instead of glass slides (GS). The purpose of this study was to gauge the perception of learning pathology using virtual slides (VS) as opposed to glass slides (GS) for medical students in Saudi Arabia. This study would help modify teaching methods with the advancement of the application of newer methods in online teaching. This two-phased study evaluated learning outcomes and perceptions in pathology online education for medical students. Using a questionnaire, Phase one analyzed second and third-year students' perceptions of the teaching methods after an online pathology course. Phase Two assessed the learning outcomes of third-year students during online practical sessions using a pretest and post-test design. Statistical data were collected using a simple additive approach. Statistical tools were used to determine the factors affecting students' perceptions. The accessibility of VS at any possible time, location, or device was the most advantageous trait of virtual learning (mean = 2.94±0.9). Students agreed the least with virtual slides as the only optimal method of learning pathology (mean = 2.25±0.9). Most enjoyed the virtual lab experience (51.7%) but still prefer both laboratory-GS and virtual-VS classes (83.5%). VS had the benefit of accessibility and efficiency. The acceptance of VS was significantly affected by the orientation prior to the online class. Findings showed that VS cannot completely replace GS and more aspects such as technical difficulties and prior VS experience should be explored.

Developing valid assessments in the era of generative artificial intelligence

Generative Artificial Intelligence (GAI) holds tremendous potential to transform the field of education because GAI models can consider context and therefore can be trained to deliver quick and meaningful evaluation of student learning outcomes. However, current versions of GAI tools have considerable limitations, such as social biases often inherent in the data sets used to train the models. Moreover, the GAI revolution comes during a period of moving away from memorization-based education systems toward supporting learners in developing the ability to apply knowledge and skills to solve real-world problems and explain real-world phenomena. A challenge in using GAI tools for scoring assessments aimed at fostering knowledge application is ensuring that these algorithms are scoring the same construct attributes (e.g., knowledge and skills) as a trained human scorer would score when evaluating student performance. Similarly, if using GAI tools to develop assessments, one needs to ensure that the goals of GAI-generated assessments are aligned with the vision and performance expectations of the learning environments for which these assessments are developed. Currently, no guidelines have been identified for assessing the validity of AI-based assessments and assessment results. This paper represents a conceptual analysis of issues related to developing and validating GAI-based assessments and assessment results to guide the learning process. Our primary focus is to investigate how to meaningfully leverage capabilities of GAI for developing assessments. We propose ways to evaluate the validity evidence of GAI-produced assessments and assessment scores based on existing validation approaches. We discuss future research avenues aimed at establishing guidelines and methodologies for assessing the validity of AI-based assessments and assessment results. We ground our discussion in the theory of validity outlined in the Standards for Educational and Psychological Testing by the American Educational Research Association and discuss how we envision building on the standards for establishing the validity of inferences made from the test scores in the context of GAI-based assessments.