Educational Assessment Research Articles

Assessing the impact of environmental education in a critical orangutan landscape in West Kalimantan, Indonesia.

Gunung Palung National Park (GPNP) and the surrounding region in West Kalimantan, Indonesia, host a significant population of Critically Endangered Bornean orangutans (Pongo pygmaeus wurmbii) amidst various conservation challenges, including habitat destruction, the illegal pet trade, and human-wildlife conflict. The Gunung Palung Orangutan Conservation Program (GPOCP) employs diverse strategies, including environmental education, to address these challenges. Environmental education helps to connect local students with conservation efforts, educate them about endangered endemic species, and highlight the relationship between human behavior and ecosystem health. Environmental education is critical in fostering sustainable behaviors and raising awareness about conservation issues, particularly in areas where human activities threaten primate biodiversity. Such programs can promote long-term conservation outcomes by engaging communities and empowering individuals to take action. Here, we review the effectiveness of GPOCP's environmental education initiatives, focusing on school visits conducted between 2016 and 2023 that concentrated on orangutan conservation. We analyze data from standardized pre- and post-activity surveys administered to students participating in puppet shows and lectures. Results indicate a substantial increase in knowledge and positive shifts in attitudes toward orangutan conservation among students. However, challenges remain in evaluating long-term behavioral changes and the relative contribution of different conservation interventions. Future efforts will explore tailored educational approaches and impact assessments to enhance conservation outcomes and foster community engagement in safeguarding Bornean orangutans and their habitat.

Using A+ Inquiry as a Framework for Exploring Faculty Needs Related to Program Assessment Workload

This paper begins by establishing the A+ Inquiry model as a theoretical lens for assessing needs related to program assessment workload by demonstrating its alignment with elements of five published frameworks associated with higher education assessment. Then, it uses the model as a frame of reference to explore faculty needs related to program assessment workload. The study examines faculty perceptions associated with five areas related to program assessment: time commitment, processes, impact, barriers, and recommendations. Quantitative and qualitative data were collected by administering an online survey to all faculty across the institution. Results revealed an average amount of time that faculty spend on program assessment efforts, tasks in assessment processes reflecting the highest and lowest levels of faculty satisfaction, a gap between perceptions of potential and actual impact of program assessment, four themes related to barriers inhibiting program assessment, and four themes related to recommendations for improving program assessment.

Measuring Spatial Ability: Analysis of Spatial Ability Test for Gulf State Students Using Item Response Theory

This study evaluates the psychometric properties of the spatial ability test using the three-parameter logistic model within item response theory. The final version of the scale comprised 29 dichotomous items, administered to a sample of 2,694 male and female students from grades 5 and 6 across schools in the Arab Gulf region. The test adhered to the three-parameter model, satisfying the assumptions of unidimensionality and local independence. The item difficulty parameters ranged from -1.541 to 1.735, discrimination parameters spanned from 0.419 to 5.252, and guessing parameters varied between 0.00 and 0.346. With a marginal reliability coefficient of 0.86, the scale demonstrated strong stability. These findings indicate that the test items align with established measurement principles, supporting the spatial ability test as a valid and reliable assessment tool for measuring spatial abilities in the Gulf region. The results have important implications for educational assessment in the Arab Gulf and could guide the development of similar assessments in other educational contexts. Further research is recommended to improve the test’s precision and explore its application in diverse educational settings.

Accuracy and reliability of large language models in assessing learning outcomes achievement across cognitive domains.

The advent of artificial intelligence (AI), particularly large language models (LLMs) like ChatGPT and Gemini, has significantly impacted the educational landscape, offering unique opportunities for learning and assessment. In the realm of written assessment grading, traditionally viewed as a laborious and subjective process, this study sought to evaluate the accuracy and reliability of these LLMs in evaluating the achievement of learning outcomes across different cognitive domains in a scientific inquiry course on sports physiology. Human graders and three LLMs, GPT-3.5, GPT-4o, and Gemini, were tasked with scoring submitted student assignments according to a set of rubrics aligned with various cognitive domains, namely "Understand," "Analyze," and "Evaluate" from the revised Bloom's taxonomy and "Scientific Inquiry Competency." Our findings revealed that while LLMs demonstrated some level of competency, they do not yet meet the assessment standards of human graders. Specifically, interrater reliability (percentage agreement and correlation analysis) between human graders was superior as compared to between two grading rounds for each LLM, respectively. Furthermore, concordance and correlation between human and LLM graders were mostly moderate to poor in terms of overall scores and across the pre-specified cognitive domains. The results suggest a future where AI could complement human expertise in educational assessment but underscore the importance of adaptive learning by educators and continuous improvement in current AI technologies to fully realize this potential.NEW & NOTEWORTHY The advent of large language models (LLMs) such as ChatGPT and Gemini has offered new learning and assessment opportunities to integrate artificial intelligence (AI) with education. This study evaluated the accuracy of LLMs in assessing an assignment from a course on sports physiology. Concordance and correlation between human graders and LLMs were mostly moderate to poor. The findings suggest AI's potential to complement human expertise in educational assessment alongside the need for adaptive learning by educators.

Context of assessment in competency-based nursing education: Semi-structured interviews in nursing education institutions in a low-income country

Organisational considerations are essential when proposing the implementation of an educational innovation like programmatic assessment. Currently, most recommendations by regulatory bodies to adopt curriculum reforms do not take into consideration the characteristics and context of low-income countries. This paper explores the contexts of four nursing education institutions in a low-income country that recently adopted the programmatic assessment approach in their programmes. The study was underpinned by the environment phase of the Design Science Research framework and used semi-structured interviews to collect data. Deductive thematic analysis was performed to map the data to three themes: the people; the organisation in terms of culture, structure and strategies; and the state of technology. The results revealed that organisational contexts in terms of the people, the organisation culture, structure and strategies and the state of its technology are essential in the implementation of programmatic assessment. Successful implementation requires a shift in organisational paradigms in terms of assessment cultures while affording maximal support and education to enhance the capabilities of students and educators experiencing the change. Collaboration with other institutions at advanced stages of programmatic assessment implementation is essential. Finally, continuous professional development for all stakeholders is essential for sustainable implementation of the programmatic assessment approach.

Simulation in podiatry teaching and learning: A scoping review.

In podiatry, there are a variety of clinical tasks that require precision and skill and it is expected that clinicians will obtain these skills during their training. Simulation is a dynamic teaching tool used in healthcare to enhance skill and knowledge acquisition. Currently, the extent and nature of the research on the use of simulation in podiatry teaching and learning are not clear. A scoping review was conducted to identify the extent and nature of research activity on the use of simulation in podiatry teaching and learning and identify gaps in the existing literature. Any research relating to simulation use in podiatry teaching including various designs and focusing on simulations aimed at improving podiatry teaching or learning were eligible for inclusion. A systematic search was conducted on February 14, 2024 of the following databases: Embase (via Embase.com), MEDLINE (via PubMed), CINAHL, and the Web of Science. Additional papers were identified via bibliographies of included studies. Content analysis of content relating to podiatry teaching and learning was performed and grouped into broad themes, then further narrowing to six themes. A total of 21 research studies were deemed eligible for inclusion focusing on diverse aspects of podiatry simulation utilized in high-income countries exclusively. Conducted between 1997 and 2023, these studies were categorized into six key themes: skill improvement, communication and professionalism, clinical competencies and patient safety, educational enhancement, and anatomy and histology education. The simulations, carried out by or assessed for podiatry professionals, staff, or students, ranged from high-fidelity medical mannequins to low-fidelity simulations such as a grapefruit model of a diabetes-related foot ulcer. Overall, the findings suggest that simulation teaching in podiatry, whether through direct skill enhancement or through educational impact assessments, holds potential in improving competency, confidence, and educational outcomes in podiatry practice. This scoping review identified a small yet diverse evidence base for simulation modalities in podiatry education, demonstrating gaps in long-term effects and comparative effectiveness studies. It highlights the urgent need for research focused on longitudinal impacts, evaluating various simulation technologies and standardizing best practices to improve podiatry education and align with clinical and patient care needs.

Associations between prenatal alcohol exposure and early education outcomes: a matched controls study using the born in Bradford dataset.

Prenatal alcohol exposure (PAE) is associated with cognitive, behavioural, and developmental impairments throughout the lifespan of affected individuals, but there is limited evidence on how early this impact can be identified through routinely collected childhood data. This paper explores the relationship between PAE and the Early Years Foundation Stage Profile (EYFSP), a statutory teacher-based summative assessment of early development in relation to learning goals. This analysis uses the Born in Bradford dataset, a UK based cohort (n = 13,959; full dataset), which collected self-reported PAE from 11,905 mothers, with 19.8% reporting drinking alcohol at some point during pregnancy. Coarsened exact matching was conducted to examine relationships between patterns of PAE and children achieving a 'Good Level of Development' on the EYFSP, a binary variable assessed at 4-5 years of age, controlling for known confounders, including deprivation, mother's education, exposure to other teratogenic substances, and child's age at assessment. Additionally, we examined EYFSP sub-scores to identify specific developmental deficits associated with PAE.The key finding is a statistically significant association between PAE at a level of consuming 5 or more units of alcohol (equivalent to 50 ml or 40 g of pure alcohol) at least once per week from the 4th month of pregnancy onwards and lower EYFSP scores when accounting for established confounding variables. These findings highlight that the detrimental impact of alcohol during pregnancy can be identified using statutory educational assessments. This has implications internationally for prevention work, policy, and commissioning of support services for people impacted by PAE.

Peran Asessment dan Evaluasi Pendidikan terhadap Akreditasi Sekolah

Accreditation is an important thing in an effort to obtain information about the real conditions of an educational institution based on minimum standards that have been set towards targeted educational planning. This research aims to determine the role of educational assessment and evaluation on school accreditation. The research method used is descriptive qualitative which describes in detail the topics discussed in this research. The research results illustrate several roles of educational assessment and evaluation in school accreditation, namely measuring the competencies of students, supporting improving teacher quality, as evidence of effective school management and supporting curriculum development. Apart from that, this research also discusses several points, namely the characteristics of educational assessment and evaluation, namely based on a comprehensive and holistic Islamic curriculum, continuous assessment (formative assessment), centered afterlife assessment, focus on spiritual development and worship, authentic and contextual assessment, oriented towards improvement and tazkiyah. Islamic education assessment and evaluation methods are competency-based assessment, authentic assessment, formative assessment, summative assessment and evaluation of worship practices. The educational assessment and evaluation approaches are the holistic approach, affective approach, spiritual approach and authentic approach.

Introduction Strategies for English Extracurricular Masterpieces

Comprehension Passages: A Guide for English Teachers provides a comprehensive overview of strategies and methodologies to enhance reading comprehension among English language learners. It begins by emphasising the crucial role of reading comprehension in language learning and identifying the common challenges students face while dealing with comprehension passages. The article outlines the cognitive processes involved in reading comprehension, including the decoding of text and the integration of new information with existing knowledge, and discusses the common obstacles students encounter. Key teaching strategies for reading comprehension are explored in detail. These include pre - reading strategies that activate prior knowledge and set purposes for reading, during - reading strategies that maintain engagement and foster understanding, and post - reading strategies that reinforce learning and evaluate comprehension. The integration of technology and multimedia in reading lessons is highlighted, emphasising the use of digital tools and interactive content to cater to diverse learning styles and preferences. Additionally, the article discusses the importance of adapting teaching methods to accommodate various learners, underscoring the need for understanding different learning styles in the context of reading comprehension. It also delves into the role of assessment and feedback in reading education, outlining effective methods for assessing reading comprehension skills and providing constructive feedback to improve student performance. Overall, the article serves as a practical guide for English teachers, offering insights and strategies to effectively navigate the complexities of teaching reading comprehension. It aims to equip educators with the tools necessary to enhance their students' reading skills, thereby fostering a deeper understanding and appreciation of the written word.

Teachers’ perceptions of the risks and benefits of AI in higher education: A case study of ERNIE Bot

ABSTRACT The study investigates teachers’ perceptions of the risks and benefits associated with the use of generative AI in higher education, with a particular focus on ERNIE Bot. The primary objective is to identify the pedagogical and ethical implications of integrating generative AI into the educational process. The study involved 400 educators from two Chinese educational institutions. The results indicate significant changes in teachers’ attitudes following the intervention. A positive aspect is that teachers reported an improvement in the effectiveness of developing educational materials and assessment tasks, particularly highlighting the AI’s ability to generate diverse, context-dependent questions. However, concerns regarding the potential for AI-generated plagiarism and the ethical implications of data accuracy persist. The findings underscore the need for further research and the development of ethical guidelines to ensure the effective and responsible integration of AI into educational processes.

Comparing paper-based and mobile application for rank-based peer assessment in interprofessional education: before, during, and after the COVID-19 pandemic

BackgroundEducation was affected during the COVID-19 pandemic, and there was a need to adapt the learning approaches to the situation. At the University of Gadjah Mada, many essential soft skills of healthcare professionals are taught using the interprofessional education (IPE) approach on-site. Our university responded to this crisis by offering online classes and similar types of training. Post-administration of the course, a peer-assessment was conducted, and it was used to provide feedback on the work or performance of peers among students. Peer assessment was done using paper-based and a mobile application during COVID-19. This study aimed to share a best practice for the implementation of a rank-based peer-assessment application for longitudinal interprofessional education in the community setting and to compare the score distribution of a rank-based peer-assessment before and after using mobile application.MethodsQuantitative research design was used by processing secondary data on student peer assessment scores from 3 bachelor programs (medicine, nursing, nutritionist) Community and Family Health Care with Interprofessional Education (CFHC-IPE) in Faculty of Medicine, Nursing, and Public Health, Universitas Gadjah Mada. 4,790 students from 2018 to 2022 (5 years) were tracked before, during, and after the COVID-19 pandemic. The assessments carried out were offline, online, and blended learning, respectively. The assessment was carried out using a paper-based method before and during the pandemic, online-based with a mobile application was used. Thus, the peer-assessment was conducted manually or using paper-based method using a 5-rank scoring system. In 2019, the peer assessment was carried out using a mobile application and applied a 10-rank scoring system.ResultsThe rank-based peer assessment can be well implemented to make students assess their friends more objectively, with an average score 82.02 ± 8.68. The rightward shift in the distribution of scores indicates that the average score has improved after using the mobile application compared to before its use. (82.02 ± 8.68 vs. 62.39 ± 11.13, p < 0.05).ConclusionThe implementation of rank-based peer-assessment using a mobile application was well received during and after the pandemic by interprofessional undergraduate students. The online system made the assessment more objective and the average grades were seen to be better.

A multimetric health literacy analysis of phalloplasty techniques: Comparing artificial intelligence and online resources

BackgroundThis study aims to compare the informational quality of online and artificial intelligence (AI) - sources on different phalloplasty donor sites to assess reliability and accessibility. MethodsA deidentified Google search was conducted using search phrases for forearm, thigh, back, and abdomen phalloplasty. The first ten websites for each donor site were included. The same search phrases were entered into three AI platforms: ChatGPT-4o mini, Google Gemini, and MetaAI. The Simple Measure of Gobbledygook (SMOG), Patient Education and Materials Assessment Tool (PEMAT), and the DISCERN tool to evaluate readability, understandability and actionability, and quality, respectively. ResultsWebsite readability was higher than AI platforms, though not statistically significant (p=0.10). All means exceeded the eighth-grade reading level, with AI outperforming websites in understandability and actionability (p<0.0001 and p=0.006). While websites were deemed understandable, they lacked actionable content. AI scores averaged 51.9, categorized as “good quality,” whereas websites averaged 46.9, classified as “fair quality” (p=0.001). No significant differences across donor sites were found (p=0.82). ConclusionsImproving the quality of online health resources for phalloplasty donor sites is essential in enabling patients to make informed surgical decisions. The emergence of AI as an educational tool enhances the understandability and quality of online phalloplasty materials, but its overall value warrants further evaluation.

COGNITIVE DIAGNOSTIC MODELING ON ASSESSMENT IN HEALTHCARE PROFESSIONS EDUCATION: A RAPID SCOPING REVIEW PROTOCOL

Background: The fast-rising supply of healthcare professional schools imposes improvement on assessment methodologies/strategies as to ensures achievement of sufficient healthcare professional competency, professional’s own health and wellbeing, and patient safety. Ascertain which learner has or has not reached sufficiency in a specific healthcare profession must be determined. Objective: The primary objective of this article is to present the protocol of a rapid scoping review to map the literature regarding the use of Cognitive Diagnostic Modeling (CDM) on assessment in healthcare professions education. Methods: This rapid scoping review protocol is registered on Open Science Framework (OSF) and followed Joanna Brigg Institute’s (JBI) updated manual for scoping reviews, Arksey and O’Malley’s framework and adhered to STructured apprOach to the Reporting In healthcare education of Evidence Synthesis (STORIES) and Best Evidence Medical Education (BEME) guidelines. A systematic and comprehensive search across MEDLINE/PubMed, Embase, Scopus, Web of Science, ERIC, LILACS, MedEdPublish and the grey literature will be conducted without date, language and study design restrictions. This review will include studies on assessment of undergraduate, graduate, and continuing medical and health professions’ education. Study selection and data extraction will be undertaken by two independent authors and synthesized into various strategies as to ensure representation of the state of the art on the use of CMD in assessment of healthcare professional education. Results: This rapid scoping review is anticipated to map CMD applications in healthcare professional education, including for admissions, progress test, assessment, and clinical reasoning. Conclusion: The findings of this rapid scoping review protocol will contribute to ensure relevance and understanding of and gap identification on the use of CDM in assessment of healthcare professions education. Futhermore, they might help academic staff, administration, and researchers in healthcare education.

Identifikasi Pendidikan sebagai Suatu Sistem untuk Kemajuan Pendidikan Menuju Indonesia Emas 2045

Education is said to be a system because there is a function and also components of education that are interconnected and there is a relationship between each other to achieve the expected goals. This research aims to find out how the education process can be said to be a system, therefore this research focuses on aspects that support the topic of the task. The research was conducted using a qualitative method literature study. The results of this study include: (1) Education Curriculum, (2) Evaluation and Assessment (3) The Meaning of Teaching and Learning (4) Student and Teacher Duties (5) Policies and Regulations (6) Education Management in Schools.

PATHS TO RESEARCH: A LITERATURE REVIEW ON HIGH SCHOOL REFORM, LAW 13.415/2017

This article presents data and reflections on the reform of secondary education, based on a systematic literature review carried out between September 2021 and January 2022. The objective was to look for possible gaps left by the selected scientific articles, in order to understand which research paths are possible in view of the reform of secondary education. This is a qualitative, documentary investigation, based on journals from the CAPES Qualis System database (four-year period 2013–2016, the latest evaluation available at the time of this investigation), in the area of ​​education assessment. A1 journals whose databases were in Federal and State Universities were analyzed, from 2017 to 2021, using the descriptors “secondary education reform” and “new secondary education”. We have observed a large number of studies related to Law 13,415/2017, which demonstrate a multitude of research paths and themes, covering Brazil from North to South. These studies highlight the researchers' concern with cultural diversity, diverse youth, the scarcity and precariousness of public schools, and the neglect and devaluation of the teaching profession, serving as a theoretical basis for many other ongoing studies.

Pelaksanaan "Aplikasi Woka Abt" untuk Penilaian Efisiensi di Sekolah Menengah Atas

In the rapidly developing digital era, the application of technology in the field of education is a must to improve the quality and efficiency of the learning and assessment process. One of the innovations that has emerged is the use of Android-based applications for the implementation of exams and assessments. This study aims to analyze the use of the Woka ABT (Android Based Test) application in improving the efficiency of final semester assessments at SMA Negeri 2 Garut. Along with the development of technology, digital-based assessment methods are increasingly needed to replace conventional assessments that take more time and resources. This study uses a quantitative approach with an experimental method. Data was collected through surveys and interviews with teachers as well as analysis of time and costs incurred before and after the use of the Woka ABT application. The results show that the use of the Woka ABT application significantly reduces the time required for the assessment process and processing of exam results. In addition, the application also reduces paper usage and operational costs, as well as improves accuracy in assessment. Teachers and students have given positive feedback to the use of this application because of the ease of access and features that help in the teaching and learning process. The conclusion of this study is that the implementation of the Woka ABT application at SMA Negeri 2 Garut contributes positively to the efficiency of the final semester assessment. The implementation of Android-based technology in educational assessment not only improves efficiency, but also supports schools' efforts towards wider digitalization. The recommendation of this study is that other schools can consider using similar applications to improve efficiency and effectiveness in the assessment process.

A Model for Effective Implementation of Contemporary Assessment Methods in Higher Education Institutions

This review study investigates the effectiveness of assessment methods in higher education institutions, focusing on the significance of contemporary and innovative methods such as project-based assessments, portfolios, simulations, and collaborative activities. The paper offers instructions for applying efficient assessment methods, policy suggestions for institutions, and potential topics for further study. The results have consequences for improving assessment methods and equipping students for their future professions since the future of assessment in higher education is seen as fertile ground for innovations. A model for effective implementation of the contemporary assessment methods is proposed.

Study of the Relationship Between Death Anxiety and Quality of Life in Patients With Heart Failure.

This study aimed to investigate the relationship between death anxiety and QoL in a sample of Iranian heart failure (HF) patients. A cross-sectional correlational design was employed to examine this relationship in a sample of 296 HF patients. Participants were recruited using convenience sampling from teaching hospitals in Kermanshah City, western Iran, between October and December 2023. Data were collected using a demographic questionnaire, the Templer Death Anxiety Scale (TDAS), and the Minnesota Living with Heart Failure Questionnaire (MLHFQ). Participants reported moderate to high levels of death anxiety (Mean [SD] = 7.54 [2.35]) on the TDAS and moderate to poor QoL (Mean [SD] = 36.24 [12.26]) on the MLHFQ. The statistical methods used in this study included descriptive statistics, independent t-tests, one-way analysis of variance (ANOVA), Pearson correlation coefficient, multiple linear regression analysis, and the Kolmogorov-Smirnov test. All statistical analyses were performed using SPSS version 26, with a significance level set at 0.05. A significant positive correlation was found between death anxiety and QoL (r = 0.329, p < .001), indicating that higher levels of death anxiety were associated with lower QoL. These results underscore the importance of addressing death anxiety in HF patients to improve their QoL. Interventions targeting the reduction of death anxiety and the enhancement of QoL are essential for optimizing care for this vulnerable population. The study recommends the implementation of psychological interventions, support groups, comprehensive counseling, education, and regular assessments for this purpose.

Integration of simulation technology with assessment in initial teacher education

Simulated learning environments have rapidly evolved in recent years and are gaining traction as an effective tool in teacher education (Australian Institute for Teaching and School Leadership, 2023; Frei-Landau et al., 2023). This paper explores the use of virtual classroom simulation in initial teacher education at an Australian university. It considers the alignment of simulation activities with mandated learning outcomes, and the challenges in assessing simulation-based learning in ways which are fair, authentic and effective. These issues are analysed within a detailed review of how simSchool, an online virtual classroom gaming platform, has been innovatively integrated within the curriculum and assessment of three undergraduate initial teacher education courses. Each case study report includes an instructor-led evaluation of key elements of the assessment design process, enabling comparison of the different strengths of each approach. The paper concludes by summarising the salient aspects of successful integration and assessment of simulation technology in higher education. This contribution to the discourse of simulation pedagogy seeks to encourage more academics to test the affordances of simulation technology in their teaching by adopting approaches that are tailored for student cohorts and precisely aligned with learning outcomes. Implications for practice or policy: Academics should tailor simulation activities for their cohort and learning objectives. Assessment must be precisely aligned, fair, authentic and effective. Learner engagement may be increased by careful introduction and management of expectations, with clarification of learning opportunities. Ongoing evaluation of simulation activities and assessment is important to maintain high quality teaching and learning.

Multimodal Assessment in Clinical Simulations: A Guide for Moving Towards Precision Education

AbstractTechnological advances and cutting-edge data analytics approaches give new prospects for using multimodal assessments in simulation-based medical education. These multimodal approaches consider the interaction of data obtained from multiple sources and can enhance our ability to assess the educational efficacy of simulations and optimize learning experiences for each individual learner. However, several challenges could jeopardize the successful implementation of this approach. We present a practical guide for integrating multimodal assessment in clinical simulation-based education, offering educators and researchers a systematic checklist to consider before undertaking these multimodal assessment initiatives.