Assessment In Higher Education Research Articles

Designing experiential learning activities with generative artificial intelligence tools for authentic assessment

Purpose This article investigates the application of generative artificial intelligence (GenAI) in experiential learning for authentic assessment in higher education. Recognized for its human-like content generation, GenAI has garnered widespread interest, raising concerns regarding its reliability, ethical considerations and overall impact. The purpose of this study is to explore the transformative capabilities and limitations of GenAI for experiential learning. Design/methodology/approach The study uses “thing ethnography” and “incremental prompting” to delve into the perspectives of ChatGPT 3.5, a prominent GenAI model. Through semi-structured interviews, the research prompts ChatGPT 3.5 on critical aspects such as conceptual clarity, integration of GenAI in educational settings and practical applications within the context of authentic assessment. The design examines GenAI’s potential contributions to reflective thinking, hands-on learning and genuine assessments, emphasizing the importance of responsible use. Findings The findings underscore GenAI’s potential to enhance experiential learning in higher education. Specifically, the research highlights GenAI’s capacity to contribute to reflective thinking, hands-on learning experiences and the facilitation of genuine assessments. Notably, the study emphasizes the significance of responsible use in harnessing the capabilities of GenAI for educational purposes. Originality/value This research showcases the application of GenAI in operations management education, specifically within lean health care. The study offers insights into its capabilities by exploring the practical implications of GenAI in a specific educational domain through thing ethnography and incremental prompting. Additionally, the article proposes future research directions, contributing to the originality of the work and opening avenues for further exploration in the integration of GenAI in education.

Developing the Technology-Integrated Assessment Framework

The purpose of this paper is to describe the development of a new framework for understanding technology-integrated assessment in higher education based on a review of the literature using the assessment design in a digital world framework (Bearman et al., 2022) as a lens. Our review (Madland et al., 2024) revealed both congruities and incongruities between the literature and the framework, leading to the need for further work to accurately conceptualize technology-integrated assessment. In this article, we contribute to the literature on technology-integrated assessment in higher education by proposing the technology-integrated assessment framework. This paper marks an important step in extending our understanding of the factors influencing instructors who integrate technology into their assessment practice and promoting ethical and equitable approaches to technology-integrated assessment in higher education.

Technical Training of Evaluators in The Context of Self-Assessment of Courses at the University of Belas

The study addressing the self-evaluation of courses and programs in HEIs, currently considered one of the main challenges of the higher education subsystem and the executive. It is believed that the creation of a culture of evaluation in the sector could have a positive impact on the quality of teaching, increasing scientific research and extension practices. The objective of the study is to verify the technical qualifications of the members of the self-evaluation committee (CAA) and assess the extent to which they can be reflected in the results of the self-evaluation process of courses and programs; It was theoretically supported by legislation that regulates higher education. This is a qualitative approach, using semi-structured interviews. Among the results, we highlight the non-adaptation of the IES's internal regulations and processes to Presidential Decree 310/20 and the legislation that regulates assessment in higher education; the longer you remain on the commission, the better the evaluator's performance and contribution to the process; in general, evaluators are not trained in areas related to evaluation and have not even had related training; evaluators have little knowledge of the legislation that regulates the evaluation process. On the other hand, there is a need to carry out a training plan that includes the dimensions of research that involve CAA members, Managers and Administrative Technical Personnel (PTA) in the UNIBELAS self-evaluation process, as well as the need to allocate budget funds to support the process.

Global state of the art of teaching life cycle assessment in higher education

Abstract Purpose Globally, there is an increased demand for education on life cycle assessment (LCA). In response, there has been an increase in course availability, but also a lack of clarity on the comprehensiveness of these offerings and the resulting student competencies. Methods A global survey was conducted to obtain empirical evidence on teaching LCA. The survey explored the availability of LCA courses globally and the depth of the teaching, including expected core competencies and related teaching and learning workloads. A purposive sampling strategy was adopted wherein eligible participants were approached by the researchers. Results and discussion According to the survey, annually, over 10,000 students participate in more than 200 LCA courses. The results reflected the interdisciplinary nature of LCA with courses being taught across different disciplines, including engineering, chemical sciences, and economics. Estimated workload demands for achieving different competency levels were significantly lower than those estimated by an expert panel before. This may be attributed in part to respondents not accounting for the full workload beyond classroom interactions. Nonetheless, workload demands increased with competency levels. Conclusions and recommendations The results emphasize the need for a common understanding of LCA teaching with regard to content, literacy levels, and competencies to avoid false expectations of the labor and research markets in terms of available expertise. Therefore, LCA curriculum development and program planning remain significant challenges and essential tasks for the global LCA community.

AI and assessment in higher education

AI and assessment in higher education

Learner Experiences of Mobile Apps and Artificial Intelligence to Support Additional Language Learning in Education

This study examines learners’ experiences and the use of language learning applications (“apps”) as a primary source of second or additional language learning (“L2”) instruction and assessment in higher education. It purviews the integration of artificial intelligence (AI)-powered features that support technology-enhanced language learning experiences. Principles of pedagogy, heutagogy, and self-determination theory are used to inform the appropriate design and application of AI to support language learning. We examine the congruence between learner's goals with perceived outcomes following a 4-week language learning intervention using an app. A survey of n = 151 adult learners across two Canadian universities revealed: (a) apps are perceived as an engaging, convenient, and structured approach to early stages of L2 learning and (b) the integration of AI for conversation-based simulations or speech recognition would enable more adaptive, personalized L2 learning experiences. The authors discuss implications for future developments and AI uptake for language learning apps.

Reflective writing as summative assessment in higher education: A systematic review

Reflective Writing as summative assessment has gained popularity across a wide range of subjects in Higher Education. This systematic review searched three cross-disciplinary databases and analysed twenty-two primary research papers published between 2007 and 2022 to investigate (1) the reasons for setting reflective writing as summative assessment, (2) support offered to students engaging in this kind of assessment, (3) student and (4) staff experiences of reflective writing as summative assessment. Using descriptive coding methodology, the review found reflective writing to be used mainly as part of larger assessment tools in professional degree programmes to foster employability and encourage students to reflect on professional practice. Support was provided through specific frameworks, exemplars, feedback and workshops, and when used to foster the incremental development of reflective writing skills in students led to positive experiences. However, the review also highlights a number of issues relating to lacking assessment literacy among students and staff. Moreover, the personal nature of reflection and power dynamics between students and markers can lead to performative instead of genuine reflection and can call into question the validity of reflective writing as summative assessment.

Exploring teachers' (future) digital assessment practices in higher education: Instrument and model development

AbstractDigital technologies are increasingly used in assessment. On the one hand, this use offers opportunities for teachers to practice assessment more effectively, and on the other hand, it brings challenges to the design of pedagogically sound and responsible digital assessment. There is a lack of validated instruments and models that explain, assess and support teachers' critical pedagogical practice of digital assessment. This explorative work first develops and validates a survey instrument to examine teachers' digital assessment practices. Secondly, we build a model to investigate to what extent teachers' pedagogical digital assessment knowledge is a foundation for the future of digital assessment (ie, authentic, accessible, automated, continuous and responsible). A total of 219 university teachers at a large European university participated in the survey study. Factor exploratory analysis and structural equation modelling were used to validate the reliability and validity of items and internal causal relations of factors. The results show the survey is a valid and reliable instrument for assessing teachers' digital assessment practice in higher education. Teachers' pedagogical knowledge and pedagogical content knowledge of digital assessment is critical, while teachers' technological pedagogical knowledge seems to have a more limited impact on the future of digital assessment.Practitioner notesWhat is already known about this topic Digital technologies are increasingly used in assessment in higher education. Teachers are the key stakeholders in the assessment for learning. There is a limited transformative nature of digital assessment practices. What this paper adds A validated instrument to assess teachers' pedagogical practice of digital assessment in higher education. Teachers' pedagogical and pedagogical content knowledge of digital assessment is critical to the future of digital assessment. Teachers' technological pedagogical knowledge of digital assessment has a more limited impact on the future of digital assessment. Implications for practice and policy Teachers need to be supported in developing pedagogical digital assessment practices in higher education. Based on this study's outcomes, educators, institutions and policymakers can inform the design and implementation of effective and responsible digital assessment practices. The design and implementation of responsible digital assessments will enhance the quality of higher education in the digital age.

Online assessment in higher education during the COVID-19

The COVID-19 pandemic has accelerated the digital transformation of education, prompting universities worldwide to swiftly transition to online and blended learning environments. The adaptation process to online education varies depending on several factors, including the level of digitalization and technological penetration in each country, the availability of technologies at universities, and the digital skills of students and staff. Additionally, the specific field of knowledge influences the adaptation process, with differences observed between disciplines such as health sciences and humanities. In this study, we conduct an exploratory qualitative analysis on how universities have responded, specifically focusing on the coordinators of Master’s programs in Spain in the specific area of assessment. This unprecedented situation has forced educators and institutions to rethink their assessment strategies and adapt to a new educational landscape. The insights gained from this study could provide valuable guidance for future crises and contribute to the ongoing discourse on the role of digital technologies in education in general, and in assessment in particular.

Unlikely allies: ChatGPT and higher education assessment

In the context of higher education, the emergence and spread of generative AI, including Large Language Models (LLMs), is revolutionising our relationship with knowledge. From spell-check and formatting tasks to rapid literature reviews, all forms of written assessments and even assessment feedback can be produced at the click of a button. We could understandably marvel at this new technology as an equaliser (at least, for those who have access to it). Second-language learners can write with native fluency. Non-traditional learners can overcome casual discrimination and academic elitism. Working class and mature learners can transition back into higher education. All this without recourse to pernicious ‘essay mills’. Yet, what of assessment? If Generative AI writes submissions for students and then marks them for lecturers, is the assessment dead? If so, what will become of learning developers - are their roles also dead, joining both assessment and the lecture in the academic afterlife? In this opinion piece we acknowledge those concerns but balance them with a message of hope. Generative AI and learning development may seem unlikely allies but there is still time to forge an alliance in the name of impactful learning and we conclude with three recommendations on how this might be achieved.

The use of a project management tool in distance education to enhance students’ engagement in group work

Group work is a widely embraced learning and assessment method that holds considerable importance in both in-person and online settings in higher education. However, group assignments introduce distinctive challenges, including unequal contributions, team conflict and divergent work ethics, leading to a pervasive reluctance among students to engage in university-level group work. Despite these concerns, group work continues to be a vital assessment approach for students, equipping them with essential skills for professional environments that prioritise teamwork and collaboration. In this opinion piece, I explore students' reluctance to engage in group work and introduce strategies and tools to transform unproductive group work into a collaborative learning experience. The discussion outlines implications for academics and learning developers, contributing to the ongoing discourse on collaborative tools in higher education assessments. With a primary focus on the challenges faced by distance-learning students, the opinion piece delves into the potential of Trello, a collaborative project management tool, to address these challenges and enhance collaborative learning in distance education. Its purpose is threefold: (1) enhance academics’ understanding of students’ reluctance to engage in group work, (2) enrich the experience of distance-learning students by highlighting the multifaceted benefits of collaborative learning in their educational journey that extend beyond traditional classroom boundaries and (3) propose innovative solutions to educators for fostering a culture of collaboration for more effective group work.

Formative assessment in higher education: an exploratory study within programs for professionals in education

This study explores how prospective professionals in higher education can learn about and apply formative assessment methods relevant to their future educational workplaces. In the academic year 2022–23, 156 pre-service teachers, social workers, and heads of social services took part in a three-stage mixed-method study on university learning experiences involving formative assessment practices. They were exposed to self-, peer-, and group-assessment strategies. Data collected after each stage revealed participants’ perspectives on each method. Findings show that students who engaged in formative assessment comprehended assessment complexity and were motivated to use diverse assessment forms. Formative assessment proves effective for both evaluation and development, supporting higher education students in honing assessment competencies for future professional roles in educational and social sectors.

A scoping review of challenges and opportunities of assessments in higher education

The post-Covid continuation of online classes and assessments in higher education at many institutions has necessitated re-evaluating assessment practices. This research article presents a scoping literature review on the challenges and innovative opportunities regarding assessments in higher education. The contextual factors of load-shedding, unavailability of Wi-Fi, the requirement for cheat-proof online assessment, and questions with ungoogleable answers or answers that ChatGPT could not provide underlined the importance of this topic. Thirty articles were analysed using the qualitative software tool, Atlas.ti. According to the literature, the main challenge is cheating with online assessments, and the best solution provided is oral presentations students’ work. This complex issue needs further research on combining contextual challenges, the credibility of assessments and the enhancement of 4IR workplace skills in one assessment model to support assessments in higher education.

What is the student experience of remote proctoring? A pragmatic scoping review

AbstractRemote or online proctoring (invigilating) is a technology primarily used to improve the integrity of online examinations. The use of remote proctoring increased significantly as the world switched to online assessment during the COVID‐19 pandemic. Remote proctoring received negative media attention, including concerns about user privacy, discrimination and the accuracy of automated systems for detecting and reporting cheating. However, it is unclear whether these media concerns fully reflect the experiences of students. Online assessment offers a number of potential advantages to learners and education providers, and it seems likely that it is here to stay. It is essential to fully understand the learner experience of remote proctoring, with a view to ensuring it is as effective as possible while meeting the needs of all stakeholders, especially those being proctored. We undertook a scoping review of research into the student experience of online proctoring, with a pragmatic focus, aimed at developing guidance for higher education providers, based on the student experience. We reviewed primary research studies which evaluated the student experience of the use of remote proctoring for summative assessment in Higher Education. We used the Education Research Information Center database (ERIC) and Google Scholar. 21 papers were identified, from which the positives and negatives of the student experience were extracted, along with the main recommendations from the research. These were then synthesised into a series of summary recommendations by thematic analysis, by a team of researchers that included students and academic staff. We found that student experience was largely negative, influenced by concerns over privacy, technological challenges, fairness and stress. Recommendations were to include the student voice in decisions about how and why to use remote proctoring and limiting the use of remote proctoring. Working with students as partners and limiting the use of remote proctoring where possible, are key to ensuring a positive student experience.

Life Cycle Assessment in Higher Education: Design and Implementation of a Teaching Sequence Activity

The latest studies show that to achieve the Sustainable Development Goals on education, there must be a focus on adequately training higher education students. In this work, we present a study about the Life Cycle Analysis of knowledge of products and processes of engineering students. This aspect is very relevant in engineering education since it has direct implications on sustainability. The first step was to identify what the learning problems were, and taking them into account, a specific teaching sequence was designed and implemented over three academic years. Two activities, on an increasing level of complexity, of the application of Life Cycle Assessment are shown in this paper. The first one is the Life Cycle Analysis comparison between two steel and polypropylene pieces. The second one is the Life Cycle Analysis comparison between three different ends of life of a polypropylene piece: mechanical recycling, incineration, and landfill. Data on the evolution of students’ marks while solving a “one step more difficult project” throughout these courses have been collected. The results show a generalized learning by the students about Life Cycle Analysis.

ChatGPT and Assessment in Higher Education: A Magic Wand or a Disruptor?

There is a current debate about the extent to which ChatGPT, a natural language AI chatbot, can disrupt processes in higher education settings. The chatbot is capable of not only answering queries in a human-like way within seconds but can also provide long tracts of texts which can be in the form of essays, emails, and coding. In this study, in the context of higher education settings, by adopting an experimental design approach, we applied ChatGPT-3 to a traditional form of assessment to determine its capabilities and limitations. Specifically, we tested its ability to produce an essay on a topic of our choice, created a rubric, and assessed the produced work in accordance with the designed rubric. We then evaluated the chatbot’s work by assessing ChatGPT’s application of its rubric according to a modified version of Paul’s (2005) Intellectual Standards rubric. Using Christensen et al.’s (2015) framework on disruptive innovations, our study found that ChatGPT was capable of completing the set tasks competently, quickly, and easily, like a “magic wand”. However, our findings also challenge the extent to which all of the ChatGPT’s demonstrated capabilities can disrupt this traditional form of assessment, given that there are aspects of its construction and evaluation that the technology is not yet able to replicate as a human expert would. These limitations of the chatbot can provide us with an opportunity for addressing vulnerabilities in traditional forms of assessment in higher education that are subject to academic integrity issues posed by this form of AI. We conclude the article with implications for teachers and higher education institutions by urging them to reconsider and revisit their practices when it comes to assessment.

A Systematic Umbrella Review on Computational Thinking Assessment in Higher Education

Computational Thinking (CT) is considered a core 21st century digital skill. The aspect of assessment is crucial and knowing what, who, when, how, and where to assess is important for assessment design. In this study, we conducted an umbrella review to gain insights regarding CT assessment in higher education. In total, we analyzed 11 reviews, focusing on: (1) bibliographical and methodological characteristics of the reviews; (2) aspects relevant of assessment design, including a) assessed constructs, b) applied assessment methodologies, and c) assessment contexts. Our findings suggest an increased attention on this topic. However, hardly any reviews reasoned the selection of their review methodology, and most of the reviews did not thoroughly examine existing reviews. Regarding assessment design aspects, most reviews did not confine their scope to higher education; however, findings on interventions and educational settings show commonalities. We identified 120 unique assessed constructs and around 10 types of assessment methods. Though a combined use of distinct assessment methods is suggested in reviews, guidelines for appropriate assessment design are yet to be constructed. Based on the findings, we argue that it is necessary to explore different combinations of assessment design in various contexts to construct assessment guidelines.

Building Academic Integrity and Capacity in Digital Assessment in Higher Education

The rapid spread of the COVID-19 pandemic in the first half of 2020 disrupted and changed higher education across the world, and into the future. Campuses were shut down, almost overnight. International and State borders were closed and business models that relied heavily on high-paying international students collapsed. University leaders and academics were forced to find new ways of attracting, engaging with, and retaining students. This paper describes a project that was undertaken in Australia in 2021 which investigated the implications of, and scope for online assessment in this ‘new virtual world’ of learning and teaching in higher education. After extensive research and consultation, the project developed a Digital Assessment Framework dubbed DASH C21, which stands for Digital Assessment Stretching Horizons for the 21st Century. The Framework is based on a set of underpinning principles and values; the Inputs. The Inputs feed into four Dimensions. These Dimensions are Practices and Pedagogies, Strategies, Emerging Technologies and Stretching Horizons. The Outputs are a series of authentic, innovative, experiential and forward looking, digital assessments, reinforced by academic integrity values. This paper will be of particular interest to higher education senior managers, academics, learning and teaching specialists, staff professional developers and curriculum designers. Keywords: Digital Assessment Framework

Learning Assessment in Noncredit Continuing Education

While program- and institution-level learning assessment in higher education has received substantial attention, much less emphasis has been placed on the assessment of learning in noncredit continuing education in higher education institutions. Learning assessment is sparse in this area because these learning experiences may be shorter in duration than credit courses, are commonly taught by field experts, and there are limits to the time and resources the higher education institution or other provider invests in conducting learning outcome assessment. This paper describes assessment recommendations in the noncredit continuing education space, with concrete examples in three areas: learning outcome statements, surveys with learning outcome-focused questions, and direct assessment of learning.

UK Research Excellence Framework (2021) Reform and Implications for China’s Implications for Research Assessment in Higher Education

UK Research Excellence Framework (2021) Reform and Implications for China’s Implications for Research Assessment in Higher Education