Academic Assessment Research Articles

Evaluating the Impact of an AI Critical Friend on Student Research Proposals, Critical AI Literacy, and Transparent Collaborative Assessment Practices

This paper presents a work-in-progress report on an innovative approach to integrating generative AI capabilities into the assessment process, focusing on research proposal development. Specifically, the study explores the impact of incorporating an AI-powered "critical friend" that provides targeted feedback and guidance to students as they craft their research proposals. The aim is to evaluate how this AI-powered support system affects students' critical AI literacy, the quality and transparency of the collaborative assessment process, and, ultimately, the strength of the final research proposals. Through a mixed-methods approach combining surveys, focus groups, and analysis of student work and interactions with generative AI, the researchers seek to uncover key insights on the benefits, challenges, and ethical considerations of leveraging AI to enhance the research proposal development experience. The findings from this study will contribute to the growing body of literature on the role of AI in academic assessments and provide practical recommendations for educators and institutions looking to responsibly integrate these emerging technologies.

Exploring the integration of Generative AI in assessment in a tertiary context

The unprecedented advancements of Generative Artificial Intelligence (GenAI) tools have generated controversies surrounding their potential for transforming educational practices, and democratising knowledge sharing, while acknowledging risks to current educational practices. In order to understand the potential and risks of GenAI tools on educational practices, recent research point to the need to develop student and teacher AI literacy skills and to investigate the impact of GenAI integration in learning and assessment practices. In addressing the call for more research in the field, this study reports on the integration of GenAI tools in an academic assessment in a postgraduate course for pre-service language teachers at an Australian university. Data were collected from students’ reflections of their adoption of GenAI tools for planning their assessment and from qualitative surveys. Thematic analysis was employed to identify students’ perceived challenges and benefits in adopting GenAI for essay writing. Pre-service teachers recognised benefits in adopting GenAI for planning and generating ideas for academic writing but recognised the importance of mitigating risks created by inaccuracies or biases in content. The findings confirm the importance of transparency in the integration of GenAI and developing student awareness of and training in ethical use of GenAI for higher education.

Demystifying the Power of Generative Artificial Intelligence Tools in Higher Education

Artificial intelligence represents an emerging frontier for higher education institutions, potentially personalising learning, automating tasks, and supporting student outcomes. This study examines international students’ perceptions of the recent proliferation of generative artificial intelligence tools in the context of academic learning and assessment. The study involved N = 223 students from three different higher education institutions located in Australia, Germany and Italy. The focus was on the student’s competence in artificial intelligence and their perception of six different generative artificial intelligence tools concerning learning and assessment. The findings suggest that the dimensions of competence in artificial intelligence vary considerably and that students from different countries have a comparable level of competence in artificial intelligence. Further findings indicate that the expected support of generative artificial intelligence tools for learning and assessment is perceived differently. This study highlights the need for increased pedagogical attention to artificial intelligence, bridging the gap between students' enthusiasm and technical knowledge. It suggests that effective integration of generative artificial intelligence tools should also prioritise the development of critical thinking and comprehension skills over content generation.

Differences in the reading performance of Texas grade 4 boys enrolled in Title I, part A and non-Title I, part A schools

In this investigation, the reading performance of Texas Grade 4 boys was compared by their enrollment or non-enrollment in Title I, Part A schools. Data were obtained from the Texas Education Agency Public Education Information Management System for all Grade 4 boys in Texas who took the State of Texas Assessment of Academic Readiness assessment in the 2016-2017, 2017-2018, and 2018-2019 school years. Inferential statistical analyses revealed the presence of statistically significant differences in all three school years. Boys enrolled in non-Title I, Part A schools outperformed boys enrolled in Title I, Part A schools in all three Reading Reporting Categories and in all three grade level standards. Recommendations for research and implications for policy and practice are suggested.

텍스트 마이닝 기법을 활용한 국어교육 평가 연구의 동향 분석

Objectives This study analyzed KCI-registered academic articles using text mining techniques to interpret the trends of research related to language education evaluation. Methods For this purpose, a total of 206 articles from January 1, 2000 to July 15, 2024 were collected from RISS and KCI, and after supplementing the abstracts, data preprocessing was performed, and finally, a total of 1,955 words were targeted for topic modeling and network analysis. Results As a result of the analysis, 10 topics were extracted and themes were extracted by topic. The themes of each topic in the research on Korean language education assessment are: improvement of Korean language teaching through assessment, school-centered Korean language assessment system, theoretical approaches to the content of Korean language assessment, development of Korean language assessment items, domain-specific characteristics and content of Korean language assessment, Korean language writing assessment, process- oriented Korean language assessment, national-level Korean language academic achievement assessment, importance and implementation of Korean language performance assessment, and assessment expertise of Korean language teachers. Conclusions The research on assessment in Korean language education tends to focus on changes in Korean language assessors' perspectives and teaching innovations, the characteristics of assessment in different areas of Korean language, research centered on writing assessment, the expansion of national-level assessment to unit school-level assessment, and theoretical research on assessment in Korean language education based on practicality.

Building tomorrow's learning landscape through AI integration and development in higher education

Birmingham City University (BCU) has responded proactively to the evolving landscape of generative artificial intelligence (GenAI) in higher education by creating written guidance and workshops for staff and students. This initiative addresses prevalent concerns surrounding AI in education and is rooted in a sector report assessing the positions of higher education institutions (HEIs) and offering guidance on effective integration of GenAI use in teaching and assessment. Our project carries several impactful aspects, including written guidance and workshops for staff and students, which reflects BCU’s dedication to addressing common concerns surrounding GenAI in education. Furthermore, the multi-agency collaborative approach brings diverse perspectives to the table. The different perspectives enable us to tailor our learner development offer through knowledge of curriculum and assessment in a particular discipline and whether that discipline is more susceptible to GenAI use than others. The insights from learner developers in creating student guidelines ensure the project is attuned to the needs and concerns of the student body, promoting a student-centred approach. This session focused on an institutional take on GenAI through the collaborative development of staff and student guidance at BCU. The presentation outlined the new AI staff and student guidance project and discussed how effective integration of GenAI education has been implemented at BCU. It also noted positive influences of GenAI through our new staff and student workshops which bring together existing and new knowledge around academic integrity and assessment literacy. Finally, it highlighted next steps around AI integration for the Education Development Service at BCU.

A literature review of "lawful" text and data mining.

Text and data mining (TDM) is a process, typically automated, that looks for patterns in data that may otherwise remain unnoticed. In a world where data driven solutions play a progressively more important role, TDM has become a vital tool in sectors ranging from medicine, to commerce, gaining widespread attraction. Nevertheless, a variety of regulatory frameworks not always specifically attuned towards regulating TDM continue to apply concurrently. The literature within the context of regulatory frameworks governing TDM is a fragmented piecemeal of valuable insights into what "lawful" TDM resembles. This literature review adopts a grounded theory approach analysing 88 pieces of literature, collating views regarding "lawful" TDM, ultimately providing a holistic assessment of academics' and practitioners' views and opinions regarding the regulatory framework which governs TDM. A total of 7 categories were identified and each of these are analysed. Tables are provided in the Appendix (accessible here: https://doi.org/10.5281/zenodo.12654691)highlighting which scholarly works were used for each section of the literature review, but also how those works were used. It is ultimately concluded that the regulatory frameworks that apply to users conducting TDM are multifaceted, and ever-changing on a case-by-case basis. There is an ever-growing need for a holistic interpretation of the regulatory frameworks which apply, creating a map which would allow for users conducting TDM to navigate this complex web of legal rules.

Impact of COVID-19 pandemic on medical and non-medical students at a private university in Malaysia

Background: Worldwide the COVID-19 pandemic affected people in various ways. Despite all the constraints students were in a unique position of being thrown into a lockdown situation where they had to learn from home and online. This had its own repercussions, and our aim was to study the effect of learning, as well as the mental and financial impact it had on the students at this private university. Methods: Using a pre-validated google questionnaire, we conducted a cross-sectional study amongst the undergraduate students from the various faculties. We used a modified DASS-21, with 15 questions rated on a Likert’s scale to assess mental health. Academic assessment and the financial impact assessment had 8 questions each. Sample size was 177. Cronbach’s for overall internal consistency was 0.860. Results: We had 208 respondents, where 66.3% were female and 62% Chinese. The mean score for medical students (mean=38.3, SD=12.9) was slightly higher than non-medical students (mean=37.3, SD=11.6). Overall non-medical students faced more challenges than the medical students academically as seen in poor contact with lecturers (chi square 10.73, p=0.030), their home environment was not conducive for learning (chi square 4.38, p=0.036) and have significantly more distraction from family members (chi square 4.87, p=0.027). Financially, a significant number of non-medical students struggled to pay for food expenses (chi square 12.92, p≤0.001). Conclusions: There was no significant difference between medical and non-medical students on mental health issues. Non-medical students were impacted more academically and financially.

The blue sky of AI-assisted language assessment: autonomy, academic buoyancy, psychological well-being, and academic success are involved

Artificial intelligence (AI) transforms the educational landscape by radically changing how lessons are taught and students are evaluated. In light of these facts, it is critical to consider learners’ emotional and mental wellness as the building blocks of education. Given these facts, it is imperative to consider learners’ emotional and mental well-being as the fundamental components of education. The key goal of this investigation was to investigate the relationships between learner autonomy (LA), academic buoyancy (AB), psychological well-being (PW), and academic success (AS) in Intelligent Computer-Assisted Language Assessment (ICALA). The current study was conducted on a cohort of 391 students registered in courses at private language institutions in Al-Kharj, Saudi Arabia, who were at the intermediate proficiency level. The following instruments were administered: the Academic Buoyancy Scale, the learner autonomy instrument, the learner psychological well-being, and the academic assessment. Then, to evaluate data and quantify the interaction between LA, AB, PW, and AS, confirmatory factor analysis (CFA) and structural equation modeling (SEM) were applied. The findings underscored the significance of LA and AB in achieving a balance in the PW of EFL pupils, particularly when utilizing ICALA for language acquisition. Additionally, the instructional implications of this study and potential avenues for future research are extensively investigated.

Thematic control and criteria-based assessment of foreign language writing skills using artificial intelligence technologies

Importance. Currently, there is a tendency that when teaching a foreign language, teachers are increasingly resorting to the use of artificial intelligence technologies to plan training sessions, generate educational content, as well as to conduct automated testing of formed communication skills. The monitoring of academic achievements and assessment is one of the key components of the organization of the educational process. Traditional methods of control and assessment require significant time and labor-intensive costs from the teacher, while artificial intelligence technologies make it possible to simplify and automate routine tasks: check tests and written papers, analyze them and identify mistakes, provide feedback. Thanks to the integration of natural language processing (NLP) technologies into chatbots and adaptive intelligent learning systems, it becomes possible on a daily basis to check texts created by students, evaluate them from the point of view of grammatical and lexical correctness, as well as identify stylistic and factual errors in real time. The purpose of this work is to test the applicability of artificial intelligence technologies for thematic control and criteria-based assessment of educational achievements using the example of productive and reproductive writing.Research Methods. In carrying out this study, the following groups of methods were used: theoretical methods aimed at familiarizing with scientific and methodological literature on the topic of research, analysis and classification of theoretical and methodological material for conducting thematic control and criterion assessment in foreign language classes, as well as empirical methods that allowed modeling pedagogical control and assessment processes using artificial intelligence technologies, observation, analysis and description of the results obtained.Definition of Concepts. The main concepts used in the study are “control of educational results” and its variety “thematic control”, “criterion assessment”.Results and Discussion. In the course of the study, various types of monitoring educational results using artificial intelligence technologies were considered: preliminary, current, intermediate, thematic, final. The choice of thematic control is due to the opportunity to trace its applicability within the framework of a single lesson in a foreign language and identify the main difficulties in using this form of control. The following criteria for evaluating writing skills are used to evaluate learning outcomes using artificial intelligence technologies: a) the structure of the written text; b) compliance with the main topic; c) coherence; d) relevance; e) grammatical correctness; f) lexical correctness; g) ethics of writing and stylistic correctness.Conclusion. Artificial intelligence technologies at the present stage have a high degree of adaptability and include a wide range of software and hardware solutions that allow for such important pedagogical procedures as monitoring educational achievements and evaluation in accordance with user-defined evaluation criteria. The obtained results are proposed to be used in research devoted to the study of modern methods of monitoring educational achievements in the methodology of teaching foreign languages using artificial intelligence technologies.

The AI Assessment Scale (AIAS) in action: A pilot implementation of GenAI-supported assessment

The rapid adoption of generative artificial intelligence (GenAI) technologies in higher education has raised concerns about academic integrity, assessment practices and student learning. Banning or blocking GenAI tools has proven ineffective, and punitive approaches ignore the potential benefits of these technologies. As a result, assessment reform has become a pressing topic in the GenAI era. This paper presents the findings of a pilot study conducted at British University Vietnam exploring the implementation of the Artificial Intelligence Assessment Scale (AIAS), a flexible framework for incorporating GenAI into educational assessments. The AIAS consists of five levels, ranging from “no AI” to “full AI,” enabling educators to design assessments that focus on areas requiring human input and critical thinking. The pilot study results indicate a significant reduction in academic misconduct cases related to GenAI and enhanced student engagement with GenAI technology. The AIAS facilitated a shift in pedagogical practices, with faculty members incorporating GenAI tools into their modules and students producing innovative multimodal submissions. The findings suggest that the AIAS can support the effective integration of GenAI in higher education, promoting academic integrity while leveraging technology’s potential to enhance learning experiences. Implications for practice or policy: Higher education institutions should adopt flexible frameworks like the AIAS to guide ethical integration of GenAI into assessment practices. Educators should design assessments that leverage GenAI capabilities, while supporting critical thinking and human input. Institutional policies related to GenAI should be developed in consultation with stakeholders and regularly updated to keep pace with technological advancements. Policymakers should prioritise research funding into the impacts of GenAI on higher education to inform evidence-based practices.

Cognitive and academic outcomes in children with myelin oligodendrocyte glycoprotein antibody-associated disease.

To describe the impact of paediatric myelin oligodendrocyte glycoprotein antibody-associated disease (MOGAD) on academic and cognitive outcomes. This was an observational, retrospective, and descriptive single-centre study, carried out on a paediatric case series of children with MOGAD. A total of 51 patients were included (22 females); their median age was 8 years and the median follow-up duration was 31.1 months (interquartile range 23.5). The most frequent clinical presentation was acute disseminated encephalomyelitis (54.9%), followed by optic neuritis (35.5%). At the last follow-up, regardless of the clinical phenotype at disease onset, 39.5% of patients with MOGAD received academic and educational interventions (p < 0.05 compared to before disease onset), including academic accommodations (p < 0.05) or the need for a learning support assistant (p < 0.05). Ten patients were evaluated with the Wechsler Intelligence Scale for Children, Fifth Edition (WISC-V). The overall IQ was calculated for six patients (mean = 92); two of these patients had an IQ lower than 85. No difference was found regarding prenatal and neonatal neurodevelopmental characteristics between this cohort and the general population. MOGAD was associated with a need for academic support; lower scores were found on the WISC-V. Patients with MOGAD should receive cognitive and academic assessments to inform educational planning and support academic success.

Addressing student non-compliance in AI use declarations: implications for academic integrity and assessment in higher education

This study examines the factors driving student non-compliance with AI use declarations in academic assessments at King’s Business School, where 74% of students failed to declare AI usage despite declaration being a requirement of a mandatory coursework coversheet. Utilising the Theory of Planned Behaviour (TPB) as a framework, the research combines service evaluation survey data and semi-structured interviews to explore how attitudes, subjective norms, and perceived behavioural control influence compliance. Findings reveal that fear of academic repercussions, ambiguous guidelines, inconsistent enforcement, and peer influence are key barriers to AI use declaration. These factors complicate the declaration process, undermine transparency, and challenge academic integrity. The study extends the TPB model by highlighting the ethical and practical dilemmas posed by generative AI, which blur traditional norms of academic integrity. This research offers critical insights for policymakers, suggesting that clear, consistent, and trust-based policies are crucial in fostering ethical AI use. The findings underscore the importance of transparent communication and supportive institutional cultures to improve compliance. Ultimately, this study informs policy development by evaluating the effectiveness of declaration mechanisms and providing actionable recommendations to promote a culture of academic integrity in the evolving landscape of AI technologies.

Academic Assessments and iPads: Performance of Students from a Low Performing Urban Elementary School

AbstractTechnology in education is used to support and supplement core instruction, to help students stay engaged, and assist teachers with academic assessments. While there is support in the use of technology in instruction and intervention, researchers still urge caution when using it in assessment. Current evidence suggests that students perform differently when assessed using traditional methods compared to technology-based methods. Further, these differences may be exacerbated for students from low-income economically marginalized at-risk communities. There is a need to thoroughly explore student performance when responding to assessment materials through different modalities. The current investigation used a within subjects group design to replicate Reynolds et al. (2023) using an urban low performing population. Results demonstrate that, on average, students answered more digits correct per minute on probes administered via paper-pencil, however preferred to answer problems on the iPad using a keyboard. These results differ from the original study that used a higher performing suburban population. Future research, limitations, and implications for educators are discussed.

The Benefits of Self –Assessment

The article examines the benefits of self-assessment and emphasizes its paramount importance in the teaching process. Our work analyzes the purpose of self-assessment, which is accompanied by numerous benefits when coupled with teachers’ feedback. The primary aim of this article is to highlight the role of students in educational activities, who are equipped with the opportunity to involve in self-assessment through different approaches. Providing constructive feedback and setting adequate goals for the learning outcomes are also indispensable components in this process. Notably, students with low self-esteem often experience anxiety and depression, which is closely intertwined with academic performance and assessment. Modern educational standards offer a plethora of possibilities to integrate innovative practices into the teaching process, establishing engaging and welcoming environment for all students. Hence, constructive and appropriate self-assessment can significantly contribute to improving students’ self-esteem, sharpening their skills, and promoting academic success. The article employs various methods, including self-reflective rubrics, self-assessment questionnaires and direct observation. Using these methods proves to have positive results in students’ academic life. The final results and conclusions of our research, conducted at Bahcesehir International University Batumi (BAU) are thoroughly discussed and conceived in the study. These findings underline the powerful impact of self-assessment on students’ academic development, marking its role in personal and educational outcomes. Results revealed that the majority of students gained autonomy and identified their strengths and areas for improvement in the learning process.

Transforming academic assessment: The metaverse‐backed Web 3 secure exam system

AbstractMetaverse—a three‐dimensional computational environment—combines physical and virtual reality to enable social relationships and immersive experiences by mimicking real‐world scenarios. Metaverse is considered the third wave of the internet revolution (exploiting Web 3.0), leveraging upcoming technologies such as extended reality and artificial intelligence shaping a new era of human–‐machine interactions. In recent years, increased research and development on educational technologies (EduTech) based on blockchain technology has seen substantial growth of metaverse‐based solutions within the higher education context. This research aims to synergize blockchain technology and metaverse environments to conduct online exams (metaExam) in a trustworthy, reliable, and secure way. The synergy between blockchain and the metaverse brings various benefits, such as improved security, cost effectiveness, and increased efficiency in the online examination process. One of the central features of the proposed solution metaExam is to leverage cryptographic protocols via blockchain to control data access, making verification faster and protecting against misuse. Exam scores and grades are stored on a blockchain ledger using a digital signature method to enhance security. We validated the proposed solution by testing a prototype on the Ethereum platform using the Sepolia Testnet network using Microsoft Windows environment. Evaluation results indicate (i) query response time (10–50 ms), (ii) and query execution performance (CPU utilization between 1%–5%) offering computationally feasible solution. This research contributes by integrating blockchain and metaverse technologies to offer a solution metaExam that can offer improved security and immersive user experience for exam management. The proposed solution and its validation can provide insights into transforming online exams, offering a fresh perspective on addressing concerns about exam grade authenticity and verifying academic credentials in EduTech.

The Precariousness of Academic Publishing in a Digital World

The world around us is growing increasingly digital and data-intensive, affecting our lives and practices as citizens and researchers in a multitude of ways. We have to ask how we ensure that academic research remains trustworthy and transparent as digitalization disrupts our practices. This article draws attention to the multifaceted nature of the challenges early-career researchers face with academic publishing in the digital era. Thus, rather than zooming in on one aspect, and losing track of the complexity of the problem, it addresses (1) the purpose of academic publishing, (2) the type of material to be published, (3) the role and use of AI and data in research, (4) the entanglement of academic publishing and research assessment, (5) the role of Open Science, and (6) what makes early-career researchers as a group different from other researchers.

Measuring Land Preservation Success: Applying Fragstats to Preserved Farmland in Lancaster County, Pennsylvania

Edge effects are traditionally discussed in biogeography and ecosystem protection. Yet preserved farmland can also experience challenging pressures at its perimeter, such as increased development pressure and conflicts between farms and nonfarm neighbors. Academic assessments of the most common farmland preservation goal—large, contiguous blocks—have focused on the area and connectivity of preserved parcels and largely overlooked the block or shape component, which relates most strongly to these types of pressures. This study fills that gap, applying the edge-effects concept to preserved farmland for the first time and tracking the shape of preserved farmland patches in Lancaster County, Pennsylvania, a leading agricultural county in the United States. The study uses the spatial pattern analysis program Fragstats to examine preserved farmland patch size, shape, and area in Lancaster County between 2000 and 2023. Results show that although land preservation organizations in Lancaster County had success at increasing the area and contiguity of preserved land, the perimeter of farmland patches increased even more significantly, even when accounting for differences in area. Farmland patches became increasingly irregular over the study period, with meandering shapes that maximize edge. The outcome suggests that tracking edge separately from area and contiguity is important because success in one does not mean success in the others. Information on edge and shape can help land preservation organizations prioritize parcels that reduce the edge effects that make it more expensive to purchase easements and more challenging for farmers to keep farming.

Another Traveler Design In Southern Patagonia: Long Edges (Knives and Side-Scrapers) Manufactured on Bifacial Blanks

ABSTRACT New information is presented on the particular design of side-scrapers and knives, referred to as long-edge artifact on biface (LEAB), and its spatial and temporal distribution in southern Patagonia is explored. The analysis considers morphological and technological characteristics, the presence of four specific attributes (overshot, platform, end thinning, and domes), and functional information at a microscopic level. A total of 101 lithics originating from academic investigations, private collections, and environmental impact assessments from different locations around the Santa Cruz province (Patagonia, Argentina) are studied. The distribution of LEAB reaches different sectors of steppe and mountain forest areas from at least 7000 years BP and into the late Holocene. In addition, LEAB manufacture among inland hunting-gathering populations and between them and maritime hunter-gatherers is recorded. These results support the transmission of information about LEAB design circulation across Southern Patagonia.

Do Confidence Ratings Reactively Modify Children's Academic Assessment Performance? Negative Answer from a Three-Year Longitudinal Study.

The reactivity effect of metacognitive judgments on first-order task performance has received increased research attention. Previous studies showed that soliciting retrospective confidence ratings (CRs) reactively enhances task performance itself, such as performance in decision making and reasoning tasks, especially for those with high self-confidence. It remains unknown whether CRs can improve students' academic assessment performance in real educational settings. The current study recruited 795 fourth-grade elementary school children to explore if making CRs reactively affects students' academic assessment performance in two main subjects (i.e., Chinese Language and Mathematics). The data were collected across six waves with half-year intervals. From Wave 2, children either provided (CR group) or did not provide CRs (no-CR group) when completing standardized academic assessments. The results showed Bayesian evidence supporting the claim that making CRs does not influence children's academic assessment performance (both the average performance across waves 2-6 and the performance in each wave) in both subjects. Furthermore, children's self-confidence did not moderate the reactive influence of CRs. The results from multilevel regression analyses re-confirmed the above conclusions. Possible explanations for the absence of the reactivity effect of CRs on children's academic assessment performance are discussed.