Educational Evaluation Research Articles

A primer: peer review process for Advances in Physiology Education.

The dissemination of discipline-focused educational scholarship advances theory and stimulates pedagogical application. The aim of Advances in Physiology Education is to publish manuscripts that advance knowledge and inform educators in the field. This primer is tailored for individuals new to manuscript reviewing, early in their careers, or experienced in reviewing research but not educational manuscripts. Peer reviewing for basic and applied science is akin to evaluating research questions and rigor in teaching and learning studies, with differences in approach and analysis similar to those between biophysics and molecular physiology or cell and integrated physiology. Our purpose is to provide an overview of the review process and expectations. The submission and peer review process involves several steps: authors submit a manuscript and the Editor assigns an Associate Editor, who then assigns peer Reviewers. Reviewers are contacted via email and can accept or decline the invitation. Reviewers evaluate the work's strengths and weaknesses and then independently submit comments and recommendations to the Associate Editor. After review, the Associate Editor collects and weighs Reviewers' comments, sometimes garners additional reviews and input, to make a recommendation to the Editor. The Editor reviews the process, comments, and recommendations to render a final decision. Both authors and Reviewers receive an email with the decision. The editorial staff assists with communication and helps track the overall process. Peer review is integral to scientific publishing, ensuring quality and rigor, and reviewing is both a privilege and a responsibility of all in the scientific community.NEW & NOTEWORTHY This mini-review offers a comprehensive and current overview of the peer review process and the qualifications required to serve as a journal reviewer for Advances in Physiology Education. The guidelines are specifically designed for early career professionals new to manuscript reviewing, as well as seasoned research manuscript reviewers who are new to educational manuscript evaluation. Peer review is a cornerstone of scientific publishing, ensuring both quality and rigor. It is both a privilege and a responsibility for all members of the scientific community.

Knowledge bloats, love builds: Paul on how we are (not) to know things

Abstract Paul's epistemology was famously mapped onto his eschatology by J. Louis Martyn, but it must be mapped also onto his ecclesiology. For Paul, knowing is bound always and indissolubly to living with others. To understand how Paul would have us know things, then, we must focus not on knowledge as such, but on epistemic practices in ecclesial communities. Whereas the Corinthians’ use of wisdom and knowledge made for fragmentation and dissolution in the body of Christ (1 Cor 1–4; 8–10), Paul would have practices with knowledge instantiate communion and care for one another, as is proper for Christ's body. Integral to theological knowing is a sense of what and whom theology is for, a sense being critically explored in recent evaluations of theological education.

Games: The WD-40 of Learning

The educational landscape has undergone a paradigm shift in recent years, particularly due to the COVID-19 pandemic. The rapid transition to remote learning environments, while ensuring educational continuity, may have inadvertently compromised the development of essential soft skills necessary for in-person interactions, such as communication, collaboration, and teamwork—skills that are vital for success in the workplace (Knight et al., 2021). While games have long been used in education (Conney & Darcy, 2020; Pagel et al., n.d.; Yuan et al., 2020), this study explores the potential of modernizing traditional game-based learning by integrating digital consoles and physical games into contemporary business classroom settings. This approach leverages the inherent appeal and engagement of games (Conney & Darcy, 2020; Hunicke et al., 2004), while fostering the development of critical soft skills. Building upon established frameworks for evaluating educational games, we explore the effectiveness of collaborative games in promoting positive learning experiences. Studies have shown that GBL can enhance motivation, knowledge retention, and problem-solving skills (Lester et al., 2023). Furthermore, collaborative games, by their very nature, necessitate effective communication, teamwork, and strategic planning among players (Pagel et al., n.d.). This project delves deeper into this aspect, investigating the potential of collaborative games to specifically address the regression of soft skills observed in the post-pandemic era. The initial pilot phase of this research focused on Pico Park on the Nintendo Switch and the Team 3 board game as potential collaborative games to cultivate soft skills. Using the MDA Framework (Hunicke et al., 2004), a well-established educational game evaluation tool, we assessed the games' ability to foster desired soft skills. Both games require effective communication, coordination, and strategic thinking, which are essential for fostering teamwork, collaboration, and problem-solving. Additionally, their accessibility and diverse formats (digital and physical) offer opportunities for students to practice soft skills in different learning environments, ultimately contributing to the research's goal of investigating the effectiveness of collaborative games in promoting positive learning experiences. The pilot involved approximately 50 students across three classes of the same course. Anonymous open-ended feedback was collected from participants using a survey questionnaire adapted from Conney and Darcy's (2020) design, which was based on the Kirkpatrick Four Level Evaluation Model (Kirkpatrick & Kirkpatrick, 1959, 1994). Participants were asked to provide their perceptions of the games, what they learnt, and the potential impact on their future group work. Preliminary findings from the pilot data are promising. Students found the games "fun", "different learning", “useful”, “refreshing”, "amazing", "engaging", "excellent for communication" and more (n=50). 98% of students (n=50) felt more confident in their ability to collaborate with others after playing the games and nearly 95% (n=50) indicated that it helped them identify potential teammates for their group assignments. Anecdotal evidence from academics and students corroborates the quantitative results, reporting increased motivation, improved team dynamics, and a more positive overall learning experience, suggesting that these activities significantly improved teamwork skills. To compliment this qualitative feedback, academics (n=3) observed a decrease in group issues and an increase in grades of around 5% for the major group assessment, bringing the average up from 60% in previous semesters to 65% in the trial period. This research aims to contribute to the existing literature on educational games and soft skills development. By demonstrating the positive impact of collaborative games on student engagement, teamwork, and soft skill development, we hope to encourage further exploration and implementation of this approach in classrooms. The findings of this pilot study highlight the potential of games to not only revitalise traditional pedagogy but also exhibits how technology can be used to address the critical need of fostering essential soft skills in a post-pandemic educational landscape.

Question Paper Generation using Bloom’s Taxonomy

The "Question Paper Generator using Bloom's Taxonomy" is a high-tech, AI-based tool designed to make the generation of varied and well-balanced assessments easier for instructors. A wide variety of input sources, including syllabus documents, banks of questions, and even historical exams, are analysed and classified by NLP and machine learning into Bloom's Taxonomy cognitive levels: Remember, Understand, Apply, Analyse, Evaluate, and Create. The AI model is actually placed in an intuitive web application with which application with which educators can readily update such parameters as total marks, question difficulty, and the distribution of cognitive levels. This tool provides a logical interface where users can create assessments centered on standards for education and elicit thinking at different cognitive dimensions. Additionally, the web application feeds back a graphic report that plots through Bloom's Taxonomy in such a way that cognitive distribution is just and transparent. With the infusion of AI in a web- based setting, the tool brings forth practical and accessible solutions for educators in automating question papers creation to enrich their assessments. This application, therefore, holds best practices in instructional design through efficiency and scalability in the approach towards educational evaluation.

Education evaluation policy in Chile: experience at a multigrade rural school as a contribution to social justice

Education evaluation policy in Chile: experience at a multigrade rural school as a contribution to social justice

Development of an AI Moral Education Assessment Tool: Identifying and Validating Key Factors through Factor Analysis

This research identified factors contributing to the development of an effective moral education evaluation tool in the context of Artificial Intelligence by using Factor Analysis By exploring these factors , the study seeks to enhance the quality of moral education and provide valuable insights for education administrators.Three hundred respondents who were actively involved in moral education programs at Yichun University in China were selected as participants in this study.Based on the factor analysis, there are five (5) factors that were extracted in the development of moral education and these are: training and support, ethical principles and guidelines, educational innovation, educational technological advancement, and acceptability and adaptability.

Portfolio Assessment and Contact-Hour-Based Evaluation (ChbE) in Architectural Education

In Architectural Education, the assessment of design courses involves the collective participation of the studio masters, the jurors, and the department board to assess the student’s ability to interpret and proffer solutions from thought processes to design proposals. Unfortunately, the assessment process is heterogeneous in the National Board for Technical Education (NBTE) accredited Schools of Architectural Technology, as institutions deploy atypical assessment scoresheets with varying variables and concomitant weightings. In most cases, such non-standard assessment variables laud students’ cognitive ability to the detriment of their affective and psychomotor persuasions in design development. This has structurally flawed the measurement of performance in the design ethos of Architectural Technology students. This issue necessitated the inception of this study towards defining the limits of assessment and evaluation protocols of design courses in Architectural Education and determining the variables that constitute criteria for assessment of portfolios. The study involved four (4) NBTE-accredited institutions that offer Architectural Technology. The methodology adopted structured questionnaires and interviews, observations, comparative analysis of scoresheets from such institutions, and archival retrieval of policy documents. Purposive sampling was adopted to administer structured questionnaires to 45 lecturers out of 63 lecturers serving in these institutions thus constituting 71.43% of the academic staff population. Results revealed poor evaluation of design miscellany in Architectural Education and the adoption of an incompatible assessment structure of design courses in the October 2020 NBTE curriculum for Architectural Technology. The study recommended the adoption of a standard Portfolio Assessment Sheet (PAS) and nouveau Contact-hour-based Evaluation module (ChbE) for measuring competence in design miscellany. A review of the assessment structure of design courses in the October 2020 NBTE curriculum and appraisal of the subsisting policy on Architectural Education were also considered imperative.

Mental Health Education Evaluation Model Based on Emotion Recognition Algorithm

Mental Health Education Evaluation Model Based on Emotion Recognition Algorithm

Establishing the Institute of Independent Evaluation of Higher Education in the Russian Federation

The paper presents the results of the search, systematization and analysis of independent higher education evaluation systems that have been developed in the country in the recent decade. The research has been carried out using standardised evaluation mechanisms that adhere to the criteria of mass scale, openness, and stability. The analysis of the independent evaluation of higher education’s present situation revealed certain inconsistencies between the legislative and regulatory framework and real practice. It should be emphasised that, while different processes have now been developed for this type of evaluation, they still require enhancement. The assessment mechanisms themselves need to undergo an external, independent evaluation to ensure their reliability, integrity, and trustworthy evidence. It has been indicated that independent evaluation procedures have jointly developed into a system: the institution of independent evaluation. This system is characterised by integrity, orderliness, and definite linkages. One may consider the aggregating technology of the aforementioned evalu-ations to be the synergistic outcome of the system’s establishment. In order to achieve a synergistic effect on the evaluation of all higher education institutions and educational programmes in the country, the authors suggest a method for aggregating the outcomes of the described evaluation procedures. The obtained research outcomes prove their applicability to the internal and external evaluation of a higher education institution’s performance and to the strategic planning of the development of both the institution and the higher education system.

Exploring The Impact and Challenges of Chatgpt on Future Primary and Secondary Education from The Perspective of Current Normal Students

Nowadays, the application of ChatGPT has received widespread attention, but there are still shortcomings at the education level. This article explores the impact and challenges of ChatGPT on future primary and secondary education from the perspective of current normal school students through a survey questionnaire method. The analysis shows that ChatGPT has penetrated the lives of normal students as a commonly used tool. Most respondents believe that ChatGPT will have a positive impact on primary and secondary education. However, regarding the educational evaluation of ChatGPT and the dependence of students on ChatGPT, respondents expressed deep concerns. Based on this, this article suggests that teachers should check the generated content and then integrate it into their curriculum design and use ChatGPT to automatically generate multi-form teaching resources. Teachers can use ChatGPT as a language companion for students' foreign language learning, but they must grasp the time for students to interact with artificial intelligence. Teachers should guide students in the use of ChatGPT, teach students to face the generated content, think from different angles, and form critical thinking. Teachers should educate students on ethics in the teaching process, emphasizing the importance of academic integrity, and at the same time, enriching the ways of evaluating students to ensure the fairness of the examination.

Education in lifestyle medicine for future physicians – a strategy to reduce the burden of cardiovascular disease

ABSTRACT There is a compelling need to improve all physicians’ knowledge on lifestyle medicine (LM). This training needs to start early during medical education. This case report describes the feasibility and effects of integrating LM into an existing core curriculum of a Swedish undergraduate medical programme. A curriculum development project was organized using a six-step model: general needs assessment, targeted needs assessment, goals and objectives, educational strategies, implementation, and evaluation. New learning objectives regarding health, prevention and lifestyle medicine were formulated throughout the five-and-a-half-year programme. Learning activities and examinations were integrated in all courses with a structured progression and with focus on preventive activities in the workplace learning. The project was evaluated using student surveys of curricular quality, teacher survey of curricular content and integration in 2020, and an external review of adherence to the national qualitative target on preventive medicine. The project was executed during 2017–2018. Course evaluations and exit poll at graduation showed an overall high-quality student rating of education in prevention and in lifestyle counselling: on average, a score of 4.1/5 and 4.1/6, respectively. The teacher survey showed that over 70% of courses had a lasting increase in LM curricular content and that this content was to a large extent integrated into the existing course material. The external review concluded that the topic was well covered and integrated into the curriculum. LM could be successfully integrated into an existing medical programme curriculum with positive effects on student learning.

New Directions and Development Models for College Student Education based on Machine Learning Classification Algorithms

For the contemporary student teaching system, the information-based education management system has played a positive role in the rapid promotion of teaching. In order to get the teaching exploration data of modern information management from the data increment on the Internet, this paper updates the technology of data mining. This paper constructs a quantitative table for teacher positions and conducts ideological and political job evaluations in the database. This paper mainly focuses on the management and construction of ideological informatization based on the [Formula: see text]-means algorithm. By using the [Formula: see text]-means algorithm to analyze the daily behavior data of students, we can discover their behavior patterns and trends, providing decision support for student management. For example, by analyzing data such as students’ internet time, social activities, and library borrowing records, we can understand their interests, hobbies, and learning habits, providing a foundation for personalized education. Cluster analysis of students’ ideological and political performance using the [Formula: see text]-means algorithm can identify student groups with different ideological tendencies. This helps schools to carry out targeted ideological and political education work, and improve the pertinence and effectiveness of ideological and political education. Through quality tracking analysis of educational evaluation, it has carried out educational informatization tracking management using an information model. In the process of judging and analyzing different subjects, it utilized the hot topics of universities for a more comprehensive information intelligent tracking management.

Comprehensive Analysis Using Probabilistic Linguistic Group Decision-Making and MEREC Technique With Sustainable Development Evaluation in Higher Education

University administrators should organize and coordinate various affairs, scientifically utilize resources such as budget, teaching equipment, and faculty, continuously improve teaching quality, and ensure the sustainable development of educational management. The sustainable development evaluation in higher education management is a multiple-attribute group decision-making (MAGDM) problem. Recently, the Exponential TODIM (ExpTODIM) and PROMETHEE technique was employed to put forward the MAGDM issues. The probabilistic linguistic term sets (PLTSs) are employed as a technique for characterizing uncertain information during the sustainable development evaluation in higher education management. In this paper, the probabilistic linguistic ExpTODIM-MABAC (PL-ExpTODIM-MABAC) technique is constructed to put forward the MAGDM under PLTSs. The MEREC technique is employed to obtain the weight values under PLTSs. Finally, a numerical example for sustainable development evaluation in higher education management is put forward to validate the ExpTODIM-MABAC technique.

Innovation Plan of Liberal Arts Education for the Cultivation of Future Convergence Talent

In this article, we looked at what changes liberal arts education should seek to cultivate future convergent talents. If major education performs the function of problem-solving, liberal arts education should be able to diagnose problems in our lives and cultivate basic academic and academic skills that are the basis for problem-solving. In order to cultivate future convergent talents, liberal arts education at universities should seek the following changes.</br>First, a convergence liberal arts curriculum should be designed. It is necessary to design and operate a thematic convergence curriculum that fits the goals of each university by utilizing the various infrastructure of the liberal arts curriculum.</br>Second, it is necessary to seek changes in the operation of liberal arts subjects through the identity of liberal arts education. Liberal arts and liberal arts education are about cultivating both knowledge and practice. Based on the performance of liberal arts education, changes should be made to the instructors in charge of liberal arts education, teaching and learning, educational evaluation, lecture size, and lecture evaluation.</br>Third, support from the Ministry of Education is needed. Liberal arts education consists of common basic studies with liberal arts ranging from humanities, society and nature. If students are compared to one tree, support for liberal arts (basic studies), which are the basis of student growth, is needed in the entire structure of ‘root-culture’, ‘flower-major’, and ‘fruit-self-achievement’ (employment, etc.).</br>Universities must bridge the gap between knowing and living. In order for universities to perform the function of fostering future convergent talents, majors and liberal arts must coexist. It is important to solve a problem, but you must be able to diagnose the problem beforehand, and the foundation for solving the problem must be solid. Liberal arts education should design a convergence curriculum to cultivate future convergent talents and operate the curriculum in accordance with the identity of liberal arts education. In order to successfully carry this out, the support of the Ministry of Education is urgently needed.

텍스트 마이닝 기법을 활용한 국어교육 평가 연구의 동향 분석

Objectives This study analyzed KCI-registered academic articles using text mining techniques to interpret the trends of research related to language education evaluation. Methods For this purpose, a total of 206 articles from January 1, 2000 to July 15, 2024 were collected from RISS and KCI, and after supplementing the abstracts, data preprocessing was performed, and finally, a total of 1,955 words were targeted for topic modeling and network analysis. Results As a result of the analysis, 10 topics were extracted and themes were extracted by topic. The themes of each topic in the research on Korean language education assessment are: improvement of Korean language teaching through assessment, school-centered Korean language assessment system, theoretical approaches to the content of Korean language assessment, development of Korean language assessment items, domain-specific characteristics and content of Korean language assessment, Korean language writing assessment, process- oriented Korean language assessment, national-level Korean language academic achievement assessment, importance and implementation of Korean language performance assessment, and assessment expertise of Korean language teachers. Conclusions The research on assessment in Korean language education tends to focus on changes in Korean language assessors' perspectives and teaching innovations, the characteristics of assessment in different areas of Korean language, research centered on writing assessment, the expansion of national-level assessment to unit school-level assessment, and theoretical research on assessment in Korean language education based on practicality.

국내 간호대학생의 핵심간호술 교육 중재의 효과: 체계적 고찰

Objectives This study systematically reviewed the effectiveness of core nursing skill training intervention programs for nursing students in Korea. Methods Literature searches were conducted using the databases RISS, KISS, DBpia, and KCI for papers published up to June 2024, with the keyword ‘core nursing skills’. A total of 130 papers were identified, and ultimately, 14 papers were selected for the literature review. Results The literature review revealed that among the research designs, 1 was randomized and 13 were nonrandomized. Core nursing skill training interventions included studies using various methods of retraining and evaluation (10 papers) and studies using various methods of self-practice and evaluation (4 papers). Retraining and evaluation methods included traditional retraining, STEPS Model retraining, senior mentoring retraining, multimedia retraining, and online learning retraining, while self-practice and evaluation methods included traditional self-practice, scenario-based self-practice, Virtual Reality-based self-practice, and self-observation methods. The most frequently applied core nursing skill was subcutaneous injection, appearing in 10 studies, followed by vital signs measurement, intramuscular injection, intermittent gastric tube feeding, and foley catheterization, each appearing in 8 studies. Outcome variables of core nursing skill training programs showed significant effects on confidence in performance (9 papers), clinical performance ability (4 papers), self-efficacy (3 papers), and knowledge (3 papers). Conclusions Korean nursing colleges have researched various intervention programs to achieve core nursing skill competency levels in accordance with the Nursing Education Evaluation Criteria. These programs have shown significant effects on performance confidence, clinical performance ability, and self-efficacy. Therefore, there is a need to develop and utilize diverse intervention programs to enhance learning motivation and satisfaction among nursing students.

KH.M.'S Educational Concept Hasyim Asy'ari's Perspective on the Jihad of Ibn Qoyyim Al Jauziyyah

This study explores the educational evaluations of KH. Hasyim Asy'ari in the context of jihad, as conceptualized by Ibn Qayyim al-Jawziyyah. The research focuses on four key aspects of evaluation: spiritual, intellectual, physical, and social, examining how these align with the broader concept of jihad. KH. Hasyim Asy'ari's holistic approach to education emphasizes the integration of spiritual, intellectual, physical, and social development to produce balanced individuals who can contribute to society and practice Islamic principles. Spiritual evaluation focuses on self-purification and closeness to Allah, intellectual evaluation emphasizes mastery of knowledge, physical evaluation highlights the importance of fitness and readiness, and social evaluation promotes contributions to the welfare of society. By aligning these educational goals with the concept of jihad, the study reveals how education serves as a powerful tool for character development, fostering a balance between spiritual growth and worldly responsibilities. The findings suggest that KH. Hasyim Asy'ari's educational model, deeply rooted in Islamic teachings, provides a comprehensive framework for preparing individuals to contribute both to their faith and society, encouraging continuous self-improvement and reflection.

Scoring Difficulty in Summary Writing Assessment: Toward the Reconstruction of Analytic Rubric

This study aims to examine whether differences exist in the factors influencing the difficulty of scoring English summaries and determining scores based on the raters’ attributes, and to collect candid opinions, considerations, and tentative suggestions for future improvements to the analytic rubric of summary writing for English learners. In this study, seven trained raters with diverse attribute backgrounds evaluated two kinds of English summaries written by Japanese university students using the analytic rubric with three evaluation items. A questionnaire was used to determine which of the three items were difficult to assess and why the raters perceived such difficulty, as well as what backgrounds and factors influenced their scoring decision-making. Moreover, through the raters’ most recent experience, candid comments were collected for developing future rubrics. The results showed that whether the evaluators’ attributes affected the difficulty of the evaluation was not clear. However, depending on the raters’ experience in teaching English/assessing summary writing, requests for improvements in the descriptors of the evaluation items and in the rubric emerged. This study proposes a tentative analytic rubric for summary writing, providing a foundation for constructing a rubric that can be used more easily by future raters. It also highlights the opinions of expert and novice teachers conducting summary evaluations in education.

A Systematic Review of Digital Competence Evaluation in Higher Education

University students’ digital skills depend significantly on educators’ proficiency, necessitating regular assessments. Tools like DigComp and the TPACK model are provided in this technological context. A systematic review, following PRISMA criteria, aims to evaluate digital competencies through globally used tools. DigCompEdu is prominent, with Spain leading the research, while unvalidated instruments from Asia highlight global disparities. This review will identify key tools and expose geographical and validation gaps, stressing the need for standardized assessments. Understanding the predominance of DigCompEdu and seeing the variation that is generated in Asia highlights the poor ability to transmit self-perceived competencies to learners.

Peer editing and writing proficiency: an experimental study in South Korean higher education

ABSTRACT Peer editing enhances critical thinking, improves student writing, ultimately leading to a more productive learning environment (Kostopoulou and O’Dwyer 2021; Lundstrom and Baker 2009, “To give is better than to receive: The benefits of peer review to the reviewer’s own writing.” Journal of Second Language Writing 18:30–43; Storch 2018, “Written corrective feedback from sociocultural theoretical perspectives: A research agenda.” Language Teaching 51 (02): 262–277; Yao et al. 2021, “How university EFL writers’ beliefs in writing ability impact their perceptions of peer assessment: Perspectives from implicit theories of intelligence.” Assessment & Evaluation in Higher Education 46 (1): 151–167). However, reservations toward peer feedback in a university EFL setting persist due to the accuracy of L2 feedback (Braine, 2003; Erdel 2023, “Student feedback literacy (SFL) profiles in an L2 writing classroom and the influence of peer feedback on the SFL features.” International Journal of Education and Literacy Studies 11 (3): 112–125.). This study utilized a pretest-posttest writing assessment to analyze differences in writing outcomes in an academic research writing course at two universities in Seoul, South Korea. The researchers also examined the accuracy of the L2 peer-editing feedback. The sampling consisted of five randomized classes of university undergraduates. Two control groups received feedback via the instructor and two experimental groups received peer-editing feedback. A fifth mixed group was divided between the control and experimental groups but within the same context. The instructors used both in-class and online feedback tools for the control and experimental groups. The analysis used Sengupta’s (1998, “Peer evaluation: ‘I am not the teacher’.” ELT Journal 52 (1): 19–28) revision taxonomy to assess peer feedback accuracy. Participants completed separate pretest and posttest writing assessments. One at the start of the semester (pretest) and one at the end (posttest). The researchers used a MANOVA to compare the pretest and posttest mean average assessment scores between the control and experimental groups. The results show that peer feedback matches the effectiveness of professor feedback for pretest/posttest scores. Researchers also assessed 156 peer-editing assignments for feedback accuracy. Peer editors demonstrated statistically significant accuracy. This study recommends using peer feedback for its benefits to L2 learners and instructors.