Technology-enhanced Assessment Research Articles

The Impact of ICT on Fostering Creativity in Business Education Pre-service Teachers: A Comprehensive Literature Review

Creativity is a global issue in higher education in the 21st century since it has shifted teaching and learning into learner-centered approaches meant for skills development. Studies show that 21st century skills foster creativity through technology and change how teaching can be approached in higher education. Since the workforce demands that students be able to apply their creativity, higher education must recognize its critical role in helping students get ready for it. Despite the benefits of creativity to students, literature shows that there is a profound gap between the knowledge and skills that students learn in higher education and the knowledge and skills they need in 21st century community and workplaces. Hence, this study explores the impact of information and communication technology (ICT) on fostering creativity in Business Education pre-service teachers as well as suggests the strategies that could assist students to develop creative skills. The study followed the systematic literature review method PRISMA in selecting studies to be included and excluded in the synthesis of the study. The process commenced by identifying the problem and screening studies to include and exclude in the study. Only 30 studies conducted between 2013 and 2023 were screened for inclusion. Based on the literature and the two-tier model of creativity, it was identified that creativity empowers learning through interactive learning tools; can be used for personalized learning; can also be used for global collaboration; and cultivates innovative minds. The paper also suggests strategies that can effectively foster creativity, such as pedagogical training and support, professional development, mentorship programs, resource accessibility, an ICT-infused curriculum, cross-disciplinary collaboration, and technology-enhanced assessment.

Developing and evaluating an e-learning and e-assessment tool for organic chemistry in higher education

A convincing e-learning system for higher education should offer adequate usability and not add unnecessary (extraneous) cognitive load. It should allow teachers to switch easily from traditional teaching to flipped classrooms to provide students with more opportunities to learn and receive immediate feedback. However, an efficient e-learning and technology-enhanced assessment tool that allows generating digital organic chemistry tasks is yet to be created. The Universities of Bonn and Duisburg-Essen are currently developing and evaluating an e-learning and technology-enhanced assessment tool for organic chemistry. This study compares the effectiveness of traditional paper-pencil-based and digital molecule-drawing tasks in terms of student performance, cognitive load, and usability—factors that all contribute to learning outcomes. Rasch analysis, t-tests, and correlation analyses were used for evaluation, revealing that the developed system can generate digital organic chemistry tasks. Students performed equally well on simple digital and paper-pencil molecule-drawing tasks when they received an appropriate introduction to the digital tool. However, using the digital tool in two of three studies imposes a higher extraneous cognitive load than using paper and pencil. Nevertheless, the students rated the tool as sufficiently usable. A significant negative correlation between extraneous load and tool usability was found, suggesting room for improvement. We are currently concentrating on augmenting the functionality of the new e-learning tool to increase its potential for automatic feedback, even for complex tasks such as reaction mechanisms.

METHODS AND TOOLS FOR ASSESSING STUDENT COMPETENCIES

The article deals with the detailed information about Methods and tools for assessing student competencies. The accurate assessment of student competencies is crucial in modern education, as it allows educators to measure not only knowledge retention but also skills, attitudes, and critical thinking abilities. This paper explores various methods and tools used for assessing student competencies, including traditional assessments, performance-based assessments, and technology-enhanced assessments. Traditional methods, such as multiple-choice tests and written exams, remain popular but are often limited in their ability to evaluate higher-order thinking skills. In contrast, performance-based assessments—such as projects, presentations, and portfolios—offer a more comprehensive evaluation by focusing on the application of knowledge in real-world contexts. Technology-enhanced tools, such as digital simulations, e-portfolios, and automated feedback systems, provide innovative ways to measure competencies while increasing engagement and personalization. The paper also discusses the importance of aligning assessment tools with learning objectives and provides insights into the challenges associated with each method, including issues of reliability, validity, and scalability. By combining multiple assessment methods, educators can obtain a holistic view of student competencies, enabling targeted support and improved learning outcomes.

Technology-enhanced assessment of surgical training

What role does technology play in ensuring that training is delivered effectively to the next generation of surgeons?

Assessing the interplay of presentation and competency in online video examinations: a focus on health system science education

BackgroundThe integration of Health System Science (HSS) in medical education emphasizes mastery of competencies beyond mere knowledge acquisition. With the shift to online platforms during the COVID-19 pandemic, there is an increased emphasis on Technology Enhanced Assessment (TEA) methods, such as video assessments, to evaluate these competencies. This study investigates the efficacy of online video assessments in evaluating medical students’ competency in HSS.MethodsA comprehensive assessment was conducted on first-year medical students (n = 10) enrolled in a newly developed curriculum integrating Health System Science (HSS) into the Bachelor of Medicine program in 2021. Students undertook three exams focusing on HSS competency. Their video responses were evaluated by a panel of seven expert assessors using a detailed rubric. Spearman rank correlation and the Interclass Correlation Coefficient (ICC) were utilized to determine correlations and reliability among assessor scores, while a mixed-effects model was employed to assess the relationship between foundational HSS competencies (C) and presentation skills (P).ResultsPositive correlations were observed in inter-rater reliability, with ICC values suggesting a range of reliability from poor to moderate. A positive correlation between C and P scores was identified in the mixed-effects model. The study also highlighted variations in reliability and correlation, which might be attributed to differences in content, grading criteria, and the nature of individual exams.ConclusionOur findings indicate that effective presentation enhances the perceived competency of medical students, emphasizing the need for standardized assessment criteria and consistent assessor training in online environments. This study highlights the critical roles of comprehensive competency assessments and refined presentation skills in online medical education, ensuring accurate and reliable evaluations.

The Effectiveness of Technology Enhanced Assessment in Malaysian Higher Education for Medical Students

Technology-enhanced assessment (TEA) means all types of assessments that use technological aid either online or offline, either during or after learning with the objective of improving or assessing the process of learning. Aims: This study assessed the effectiveness and influence of TEA on the academic performance of medical students in a Malaysian private university. A cross-sectional study was conducted and convenience sampling was applied to recruit participants. A validated online questionnaire was used and the participants were required to report their demographic characteristics, academic performance before and after using TEA, and their attitudes towards the TEA. Paired-t-test was performed to assess the difference in medical students’ mean CGPA before and after using TEA. Study findings showed that medical students have positive attitudes towards TEA and believe that TEA offers value to their learning. TEA was found to have a positive impact on the academic performance by which there is a significance difference in medical students’ mean CGPA before and after using TEA (p0.05). TEA is an effective tool that can enhance the learning process among medical students. Nevertheless, the sample studied need to be more diverse to be able to generalize the results for all medical students along different universities and countries.

Demystifying Classroom Assessment: Unleashing the Power of Technology Tools

Background: Technology is a tool for instruction, not a replacement for teachers. Teachers are crucial for our nation's future and are essential implementers and integrators of technology in classrooms. They are responsible for innovative use of technology, as it enhances efficiency and effectiveness in everyday life. ICT comprehensively combines computer and telecommunication technology. Aim: They are also commonly used for the certification of competence, to monitor educational standards, and to provide learners with feedback. The qualitative aspect of assessment and evaluation in the field of education has a cutting-edge relationship with students’ academics. Method: Teachers no doubt are saddled with many responsibilities which range from writing lesson plans and notes to instructional delivery and assessment. Hence there is a need for teachers to span to the innovation technology is offering by embracing technological-based assessment. This paper explored the roles of technology in bridging the gap between the paper-pencil assessment to technology-enhanced assessment and proffered strategies on how these assessment apps can be used in the classroom. The study revealed that there are free assessment applications teachers can use to grade students, thereby reducing their workloads. It was therefore recommended among others that training and seminars should be organised for teachers to enlighten them on technology-based assessment. Result and Discussions: Teachers need to take cognizance of the relevancy of the technological tools. It was further pointed out that while observing the relevance of the technology resources, there should be congruence between the tech tools and instructional curriculum. Other factors to be considered are the; appropriateness of the content and alignment with the learning goals. Teachers need to have clear-cut goals before selecting any technology for assessment. Among the common goals to be considered are; efficiency, cost, end-user proficiency, collaboration, and capacity. Conclusıon: Assessment is vital to the education process. Online assessment can help learners engage more effectively and flexibly in knowledge and can help teachers manage assessment more proficiently and efficiently.

Role of technology enhanced assessment in medical and dental education: A scoping review

Background: Evolving innovative, feasible and cost effective technology related with teaching and learning in context of recent COVID-19 pandemic has led the medical educationists to think about its increased use in the assessment process. Need of the hour is to develop and establish a framework in our institutions for technology enhanced assessment (TEA) efficiently. European countries were using the e- assessment before the COVID-19 pandemic so their education system was less disturbed by it as compared to the Asian countries like Pakistan. TEA can be done with a number of tools and mobile phone technology, learning management systems (LMS) and social media have played remarkable role in this regard. The objective of this scoping review was to find best practice regarding the use of TEA in context of COVID-19 pandemic according to the Best Evidence Medical Education (BEME) principles. Methods: Databases which were searched were Google Scholar, Educational Resource Information System (ERIC), Cochrane library and ScienceDirect (Elsevier). The key words and phrases which we used for search were "TEA", "Online assessment", "assessment in context of COVID-19", "Role of TEA", "Advantages and disadvantages of TEA". 1252 articles were retrieved after screening total searched articles by Mendeley software. These articles were then assessed according to the eligibility criteria and 84 articles were selected out of which finally 12 articles were included in this scoping review after assessment of quality by Qualsyst appraisal tool. Data was extracted by using data extraction form which was synthesized to create the themes by axial coding. Results: After synthesis of data following themes were identified describing the TEA. What is TEA, role of TEA. Advantages and disadvantages of TEA, measurement and implementation issues in TEA. Prime benefit of TEA is its potential to foster learning by its ability to deliver vast variety of test formats in high fidelity virtual reality situations and by providing feedback after formative assessment using number of digital tools. Major issues related to TEA are cheating, validity, quality and security of test materials. Conclusion: E-assessment tools were already being used in European countries with the intent to improve the teaching and learning activities but after the COVID-19 pandemic it became the necessity. TEA has extreme potential for improving the learning and assessment process especially in context of current COVID-19 pandemic if the available technology is used efficiently. Implementation of TEA in place of conventional assessment should be tailored according to the local requirements and context. Finally the word of caution is to be prepared to address the issues of validity, quality, cheating and security of test materials connected with this technological change.

Surgical Sabermetrics: A Scoping Review of Technology-enhanced Assessment of Nontechnical Skills in the Operating Room.

To evaluate the current evidence for surgical sabermetrics: digital methods of assessing surgical nontechnical skills and investigate the implications for enhancing surgical performance. Surgeons need high-quality, objective, and timely feedback to optimize performance and patient safety. Digital tools to assess nontechnical skills have the potential to reduce human bias and aid scalability. However, we do not fully understand which of the myriad of digital metrics of performance assessment have efficacy for surgeons. A systematic review was conducted by searching PubMed, EMBASE, CINAHL, and PSYCINFO databases following PRISMA-ScR guidelines. MeSH terms and keywords included "Assessment," "Surgeons," and "Technology". Eligible studies included a digital assessment of nontechnical skills for surgeons, residents, and/or medical students within an operative context. From 19,229 articles screened, 81 articles met the inclusion criteria. The studies varied in surgical specialties, settings, and outcome measurements. A total of 122 distinct objective, digital metrics were utilized. Studies digitally measured at least 1 category of surgical nontechnical skill using a single (n=54) or multiple objective measures (n=27). The majority of studies utilized simulation (n=48) over live operative settings (n=32). Surgical Sabermetrics has been demonstrated to be beneficial in measuring cognitive load (n=57), situation awareness (n=24), communication (n=3), teamwork (n=13), and leadership (n=2). No studies measured intraoperative decision-making. The literature detailing the intersection between surgical data science and operative nontechnical skills is diverse and growing rapidly. Surgical Sabermetrics may provide a promising modifiable technique to achieve desirable outcomes for both the surgeon and the patient. This study identifies a diverse array of measurements possible with sensor devices and highlights research gaps, including the need for objective assessment of decision-making. Future studies may advance the integration of physiological sensors to provide a holistic assessment of surgical performance.

An Overview of the Adoption of eAssessment Among Moroccan Primary School Teachers: Factors and Challenges

This study provides a comprehensive investigation of how primary school teachers in the Marrakech-Safi region of Morocco integrate information and communication technologies (ICTs) into their assessment practices. Gathering data through a questionnaire disseminated via email, the research garnered 346 valid responses, indicating a modest implementation of ICTs in assessment processes, with a small fraction of teachers applying them consistently. Interestingly, the findings demonstrate that experienced educators are more inclined to employ ICT in assessments. Despite the prevalent practice of offering eAssessment training to Moroccan primary school teachers as a means to bolster their capabilities in technology-enhanced assessment, such professional development has not been a significant factor in encouraging the use of ICT-based assessment tools among teachers who do not already utilize them. The investigation also delves into the types of eAssessment methods employed, including interactive tests, simulations, and problem-solving activities. The research underscores the necessity for additional exploration and support systems to advance effective ICT adoption in educational assessments. The research has substantial implications for the enhancement of educational assessment practices, informing policy-makers and practitioners about the current state of eAssessment and guiding future efforts to optimize the evaluation of student knowledge in a digital era.

Advancing Research and Evaluation Methods in Special Education: Enhancing Practices and Improving Outcomes

This paper discusses the importance of research and evaluation in special education and its impact on improving outcomes for individuals with disabilities. It explores common challenges in conducting research and evaluation in special education, such as access to participants, measurement and assessment, ethical considerations, limited resources, and generalizability. The paper also highlights emerging trends and innovative approaches in special education research and evaluation, including mixed methods research, technology-enhanced assessment and intervention, implementation science, and participatory research. Additionally, it offers recommendations for future research in special education, including longitudinal studies, intersectionality and inclusivity, teacher preparation and professional development, family engagement and involvement, transition and post-school outcomes, inclusive practices and environments, technology and assistive technology, culturally responsive practices, and policy and systems-level research. By addressing these areas, special education can continue to evolve and improve, leading to more equitable and effective educational experiences for individuals with disabilities.

Validating an Automated System of Evaluation for English Drafting Skill: A Case of Large-scale, High-stakes Selection of Entry-level Managerial Positions

The effectiveness of using a constructed response measure for assessing drafting ability is proven and has been used extensively in managerial selection in state-owned organizations. With the advent of online recruitment trends and technology-enhanced assessments, automated scoring has been conceived as a replacement for human scoring with the purpose of emulating the human scoring system. In the context of large-scale writing assessments, automated scoring could provide superior results to human scoring in terms of validity and reliability. In the present study, an attempt has been made to validate an automated essay scoring (AES) algorithm. The study was conducted on a sample of 11,497 randomly selected from a population of 54,392 shortlisted candidates for a national-level examination for the selection of entry-level executives in managerial positions in state-owned banks and a state-owned insurance company. The evaluation of the descriptive (constructed response) component of the examination (English composition) was carried out parallelly by four expert human raters and the AES algorithm. The parameters for evaluation were devised and made available to raters and the algorithm. Data were analysed using mean SD and Pearson correlation coefficient. Results show that the mean scores of human expert raters ( M = 14.648) and automated algorithm method ( M = 15.804) were similar. Further analysis was also undertaken to check the convergent validity of the features used in the algorithm by examining the relation of algorithm scores with sub scores from the objective test of the same construct. Results indicate a significant correlation. It can thus be said that the algorithm scoring method developed can be considered a replacement in that it can complement human expert raters in the evaluation of descriptive papers with consistent scoring and fairness, without inherent biases of inter-rater and intra-rater variation, in addition to practical benefits of speed and cost.

“Expanding our pedagogical imagination”: Faculty Experiences with Scholarly Digital Storytelling across Disciplines

Despite growing pressure to increase technology-enhanced assessment, research has repeatedly demonstrated a lack of widespread meaningful integration of technology in higher education. This is in part due to faculty reluctance which is compounded by lack of training and incentives. This qualitative research grounded in the scholarship of teaching and learning (SoTL) is based on in-depth interviews with 25 faculty internationally, across disciplines and teaching contexts, who integrated scholarly digital storytelling into their pedagogy. It examines their motivations for adopting this technology-enhanced assessment and their experiences implementing it, including increased engagement and student learning. It further explores challenges faced and lessons learned that may prove valuable in supporting the expansion of technology-enhanced assessment across disciplines.

Artificial Intelligence in Technology-Enhanced Assessment: A Survey of Machine Learning

Intelligent assessment, the core of any AI-based educational technology, is defined as embedded, stealth and ubiquitous assessment which uses intelligent techniques to diagnose the current cognitive level, monitor dynamic progress, predict success and update students’ profiling continuously. It also uses various technologies, such as learning analytics, educational data mining, intelligent sensors, wearables and machine learning. This can be the key to Precision Education (PE): adaptive, tailored, individualized instruction and learning. This paper explores (a) the applications of Machine Learning (ML) in intelligent assessment, and (b) the use of deep learning models in ‘ knowledge tracing and student modeling’. The paper concludes by discussing barriers involved in using state-of-the-art ML methods and some suggestions to unleash the power of data and ML to improve educational decision-making.

Technology-Enhanced Assessment Feedback on Preservice Teachers’ Core Teaching Practices

The teaching practicum has been characterised by “wicked feedback” problems. Feedback is essential for enhancing student teachers’ core teaching practices. Student teachers are most vulnerable when they have to present lesson segments and may not know whether they are doing it correctly. Providing actionable, timely and constructive feedback is challenging at the best of times and this became even more so during the COVID-19 pandemic. This proactive action research study aims to provide an overview of the utilization of technology-enhanced feedback (digital screen recording) on preservice teachers’ practicum assessment tasks and to report on their perceptions of, reflections on, and uptake of the feedback. A group of fifteen preservice teachers and two teacher educators from a university in the North West Province participated in the study. The findings indicate that technology-enhanced feedback motivates student teachers to engage with the feedback because it facilitates the personalization of feedback, fosters relationships between preservice teachers and teacher educators, and increases the uptake of feedback that results in improved teaching practicum performance tasks. The study informs a developing understanding of the pedagogic potential of feedback via technology and its uptake within an environment that is becoming increasingly digital. The article concludes with recommendations for exploiting the potential benefits of technology-enhanced feedback to ensure uptake and facilitate the improved teaching practice of preservice teachers within teaching practicum settings. Keywords: Technology-enhanced, Feedback, Preservice teachers, Core teaching practices, Assessment

Technology enhanced assessment: Ottawa consensus statement and recommendations

Introduction In 2011, a consensus report was produced on technology-enhanced assessment (TEA), its good practices, and future perspectives. Since then, technological advances have enabled innovative practices and tools that have revolutionised how learners are assessed. In this updated consensus, we bring together the potential of technology and the ultimate goals of assessment on learner attainment, faculty development, and improved healthcare practices. Methods As a material for the report, we used the scholarly publications on TEA in both HPE and general higher education, feedback from 2020 Ottawa Conference workshops, and scholarly publications on assessment technology practices during the Covid-19 pandemic. Results and conclusion The group identified areas of consensus that remained to be resolved and issues that arose in the evolution of TEA. We adopted a three-stage approach (readiness to adopt technology, application of assessment technology, and evaluation/dissemination). The application stage adopted an assessment ‘lifecycle’ approach and targeted five key foci: (1) Advancing authenticity of assessment, (2) Engaging learners with assessment, (3) Enhancing design and scheduling, (4) Optimising assessment delivery and recording learner achievement, and (5) Tracking learner progress and faculty activity and thereby supporting longitudinal learning and continuous assessment.

Improvement Plan on the Utilization of Mobile Applications with Technology-Enhanced Assessment (TEA) Tool

Improvement Plan on the Utilization of Mobile Applications with Technology-Enhanced Assessment (TEA) Tool

Cultivating research skills through scholarly digital storytelling

ABSTRACT A fundamental responsibility of higher education institutions, across disciplines, is to develop student research skills. The exponential growth of digital scholarship, however, challenges traditional definitions of research. There is a need for deeper understanding of the ways in which students can develop and implement academic research skills and digital research skills simultaneously. This qualitative study explored graduate student engagement in scholarly digital storytelling, a technology-enhanced assessment, across multiple disciplines at a research university in the United States. Findings suggest that students developed the essential research skills of autonomy and flexible thinking along with multimodal research skills throughout the semester-long process of creating scholarly digital stories. Implications for utilizing technology-enhanced assessments, such as scholarly digital storytelling, in higher education are discussed.

Crisis and Changes in Learning Behaviours: Technology-Enhanced Assessment in Language Learning Contexts

ABSTRACT The outbreak of COVID-19 has posed a sudden and unprecedented change to the educational system. This study investigated the impact of this sudden change on EFL learners’ perceptions of assessment in a technology-enhanced learning environment. Assessment dimensions such as affective factors, validity, reliability, practicality, security and pedagogy were explored through an online questionnaire administered among 345 Iranian EFL learners and a focus-group interview with 17 learners. In addition, through a quasi-experimental design, the study compared the learners’ academic achievements of online and in-person classrooms during the COVID-19 outbreak. The findings revealed that the flexibility of online assessment, immediate feedback and encouraging self-regulated learning are among the advantages and technical problems, technology dependence and technology literacy are among the challenges of technology-enhanced assessment. The findings indicated that while technology-enhanced assessment has a positive impact on learners’ performances, learners have various attitudes towards e-assessment that need to be addressed in future studies.

Developing student capacity to produce digital scholarship in the humanities

Despite the increased use of technology in higher education classrooms, we need a better understanding of pedagogical strategies that improve student ability to produce quality scholarly digital content in the humanities. This research was designed to examine student learning through scholarly digital storytelling, a technology-enhanced assessment. The researcher collected data during and after an interdisciplinary, graduate scholarly digital storytelling course, including student work, student reflections, and individual interviews, to examine experiences at key points throughout the learning process. The results indicate that this pedagogical approach, when carefully scaffolded alongside formative feedback and ongoing student support, can increase student capacity—including digital agency, problem-solving skills, and digital knowledge production skills—to produce scholarly digital work in the humanities. Students can also learn to understand the interplay between disciplinary learning and digital skills and the ways in which both are essential for scholarly communication within and beyond the classroom.