Technology-enhanced Assessment Research Articles

Role of technology enhanced assessment in medical and dental education: A scoping review

Background: Evolving innovative, feasible and cost effective technology related with teaching and learning in context of recent COVID-19 pandemic has led the medical educationists to think about its increased use in the assessment process. Need of the hour is to develop and establish a framework in our institutions for technology enhanced assessment (TEA) efficiently. European countries were using the e- assessment before the COVID-19 pandemic so their education system was less disturbed by it as compared to the Asian countries like Pakistan. TEA can be done with a number of tools and mobile phone technology, learning management systems (LMS) and social media have played remarkable role in this regard. The objective of this scoping review was to find best practice regarding the use of TEA in context of COVID-19 pandemic according to the Best Evidence Medical Education (BEME) principles. Methods: Databases which were searched were Google Scholar, Educational Resource Information System (ERIC), Cochrane library and ScienceDirect (Elsevier). The key words and phrases which we used for search were "TEA", "Online assessment", "assessment in context of COVID-19", "Role of TEA", "Advantages and disadvantages of TEA". 1252 articles were retrieved after screening total searched articles by Mendeley software. These articles were then assessed according to the eligibility criteria and 84 articles were selected out of which finally 12 articles were included in this scoping review after assessment of quality by Qualsyst appraisal tool. Data was extracted by using data extraction form which was synthesized to create the themes by axial coding. Results: After synthesis of data following themes were identified describing the TEA. What is TEA, role of TEA. Advantages and disadvantages of TEA, measurement and implementation issues in TEA. Prime benefit of TEA is its potential to foster learning by its ability to deliver vast variety of test formats in high fidelity virtual reality situations and by providing feedback after formative assessment using number of digital tools. Major issues related to TEA are cheating, validity, quality and security of test materials. Conclusion: E-assessment tools were already being used in European countries with the intent to improve the teaching and learning activities but after the COVID-19 pandemic it became the necessity. TEA has extreme potential for improving the learning and assessment process especially in context of current COVID-19 pandemic if the available technology is used efficiently. Implementation of TEA in place of conventional assessment should be tailored according to the local requirements and context. Finally the word of caution is to be prepared to address the issues of validity, quality, cheating and security of test materials connected with this technological change.

Surgical Sabermetrics: A Scoping Review of Technology-enhanced Assessment of Nontechnical Skills in the Operating Room.

To evaluate the current evidence for surgical sabermetrics: digital methods of assessing surgical nontechnical skills and investigate the implications for enhancing surgical performance. Surgeons need high-quality, objective, and timely feedback to optimize performance and patient safety. Digital tools to assess nontechnical skills have the potential to reduce human bias and aid scalability. However, we do not fully understand which of the myriad of digital metrics of performance assessment have efficacy for surgeons. A systematic review was conducted by searching PubMed, EMBASE, CINAHL, and PSYCINFO databases following PRISMA-ScR guidelines. MeSH terms and keywords included "Assessment," "Surgeons," and "Technology". Eligible studies included a digital assessment of nontechnical skills for surgeons, residents, and/or medical students within an operative context. From 19,229 articles screened, 81 articles met the inclusion criteria. The studies varied in surgical specialties, settings, and outcome measurements. A total of 122 distinct objective, digital metrics were utilized. Studies digitally measured at least 1 category of surgical nontechnical skill using a single (n=54) or multiple objective measures (n=27). The majority of studies utilized simulation (n=48) over live operative settings (n=32). Surgical Sabermetrics has been demonstrated to be beneficial in measuring cognitive load (n=57), situation awareness (n=24), communication (n=3), teamwork (n=13), and leadership (n=2). No studies measured intraoperative decision-making. The literature detailing the intersection between surgical data science and operative nontechnical skills is diverse and growing rapidly. Surgical Sabermetrics may provide a promising modifiable technique to achieve desirable outcomes for both the surgeon and the patient. This study identifies a diverse array of measurements possible with sensor devices and highlights research gaps, including the need for objective assessment of decision-making. Future studies may advance the integration of physiological sensors to provide a holistic assessment of surgical performance.

An Overview of the Adoption of eAssessment Among Moroccan Primary School Teachers: Factors and Challenges

This study provides a comprehensive investigation of how primary school teachers in the Marrakech-Safi region of Morocco integrate information and communication technologies (ICTs) into their assessment practices. Gathering data through a questionnaire disseminated via email, the research garnered 346 valid responses, indicating a modest implementation of ICTs in assessment processes, with a small fraction of teachers applying them consistently. Interestingly, the findings demonstrate that experienced educators are more inclined to employ ICT in assessments. Despite the prevalent practice of offering eAssessment training to Moroccan primary school teachers as a means to bolster their capabilities in technology-enhanced assessment, such professional development has not been a significant factor in encouraging the use of ICT-based assessment tools among teachers who do not already utilize them. The investigation also delves into the types of eAssessment methods employed, including interactive tests, simulations, and problem-solving activities. The research underscores the necessity for additional exploration and support systems to advance effective ICT adoption in educational assessments. The research has substantial implications for the enhancement of educational assessment practices, informing policy-makers and practitioners about the current state of eAssessment and guiding future efforts to optimize the evaluation of student knowledge in a digital era.

Advancing Research and Evaluation Methods in Special Education: Enhancing Practices and Improving Outcomes

This paper discusses the importance of research and evaluation in special education and its impact on improving outcomes for individuals with disabilities. It explores common challenges in conducting research and evaluation in special education, such as access to participants, measurement and assessment, ethical considerations, limited resources, and generalizability. The paper also highlights emerging trends and innovative approaches in special education research and evaluation, including mixed methods research, technology-enhanced assessment and intervention, implementation science, and participatory research. Additionally, it offers recommendations for future research in special education, including longitudinal studies, intersectionality and inclusivity, teacher preparation and professional development, family engagement and involvement, transition and post-school outcomes, inclusive practices and environments, technology and assistive technology, culturally responsive practices, and policy and systems-level research. By addressing these areas, special education can continue to evolve and improve, leading to more equitable and effective educational experiences for individuals with disabilities.

Validating an Automated System of Evaluation for English Drafting Skill: A Case of Large-scale, High-stakes Selection of Entry-level Managerial Positions

The effectiveness of using a constructed response measure for assessing drafting ability is proven and has been used extensively in managerial selection in state-owned organizations. With the advent of online recruitment trends and technology-enhanced assessments, automated scoring has been conceived as a replacement for human scoring with the purpose of emulating the human scoring system. In the context of large-scale writing assessments, automated scoring could provide superior results to human scoring in terms of validity and reliability. In the present study, an attempt has been made to validate an automated essay scoring (AES) algorithm. The study was conducted on a sample of 11,497 randomly selected from a population of 54,392 shortlisted candidates for a national-level examination for the selection of entry-level executives in managerial positions in state-owned banks and a state-owned insurance company. The evaluation of the descriptive (constructed response) component of the examination (English composition) was carried out parallelly by four expert human raters and the AES algorithm. The parameters for evaluation were devised and made available to raters and the algorithm. Data were analysed using mean SD and Pearson correlation coefficient. Results show that the mean scores of human expert raters ( M = 14.648) and automated algorithm method ( M = 15.804) were similar. Further analysis was also undertaken to check the convergent validity of the features used in the algorithm by examining the relation of algorithm scores with sub scores from the objective test of the same construct. Results indicate a significant correlation. It can thus be said that the algorithm scoring method developed can be considered a replacement in that it can complement human expert raters in the evaluation of descriptive papers with consistent scoring and fairness, without inherent biases of inter-rater and intra-rater variation, in addition to practical benefits of speed and cost.

“Expanding our pedagogical imagination”: Faculty Experiences with Scholarly Digital Storytelling across Disciplines

Despite growing pressure to increase technology-enhanced assessment, research has repeatedly demonstrated a lack of widespread meaningful integration of technology in higher education. This is in part due to faculty reluctance which is compounded by lack of training and incentives. This qualitative research grounded in the scholarship of teaching and learning (SoTL) is based on in-depth interviews with 25 faculty internationally, across disciplines and teaching contexts, who integrated scholarly digital storytelling into their pedagogy. It examines their motivations for adopting this technology-enhanced assessment and their experiences implementing it, including increased engagement and student learning. It further explores challenges faced and lessons learned that may prove valuable in supporting the expansion of technology-enhanced assessment across disciplines.

Artificial Intelligence in Technology-Enhanced Assessment: A Survey of Machine Learning

Intelligent assessment, the core of any AI-based educational technology, is defined as embedded, stealth and ubiquitous assessment which uses intelligent techniques to diagnose the current cognitive level, monitor dynamic progress, predict success and update students’ profiling continuously. It also uses various technologies, such as learning analytics, educational data mining, intelligent sensors, wearables and machine learning. This can be the key to Precision Education (PE): adaptive, tailored, individualized instruction and learning. This paper explores (a) the applications of Machine Learning (ML) in intelligent assessment, and (b) the use of deep learning models in ‘ knowledge tracing and student modeling’. The paper concludes by discussing barriers involved in using state-of-the-art ML methods and some suggestions to unleash the power of data and ML to improve educational decision-making.

Technology-Enhanced Assessment Feedback on Preservice Teachers’ Core Teaching Practices

The teaching practicum has been characterised by “wicked feedback” problems. Feedback is essential for enhancing student teachers’ core teaching practices. Student teachers are most vulnerable when they have to present lesson segments and may not know whether they are doing it correctly. Providing actionable, timely and constructive feedback is challenging at the best of times and this became even more so during the COVID-19 pandemic. This proactive action research study aims to provide an overview of the utilization of technology-enhanced feedback (digital screen recording) on preservice teachers’ practicum assessment tasks and to report on their perceptions of, reflections on, and uptake of the feedback. A group of fifteen preservice teachers and two teacher educators from a university in the North West Province participated in the study. The findings indicate that technology-enhanced feedback motivates student teachers to engage with the feedback because it facilitates the personalization of feedback, fosters relationships between preservice teachers and teacher educators, and increases the uptake of feedback that results in improved teaching practicum performance tasks. The study informs a developing understanding of the pedagogic potential of feedback via technology and its uptake within an environment that is becoming increasingly digital. The article concludes with recommendations for exploiting the potential benefits of technology-enhanced feedback to ensure uptake and facilitate the improved teaching practice of preservice teachers within teaching practicum settings. Keywords: Technology-enhanced, Feedback, Preservice teachers, Core teaching practices, Assessment

Technology enhanced assessment: Ottawa consensus statement and recommendations

Introduction In 2011, a consensus report was produced on technology-enhanced assessment (TEA), its good practices, and future perspectives. Since then, technological advances have enabled innovative practices and tools that have revolutionised how learners are assessed. In this updated consensus, we bring together the potential of technology and the ultimate goals of assessment on learner attainment, faculty development, and improved healthcare practices. Methods As a material for the report, we used the scholarly publications on TEA in both HPE and general higher education, feedback from 2020 Ottawa Conference workshops, and scholarly publications on assessment technology practices during the Covid-19 pandemic. Results and conclusion The group identified areas of consensus that remained to be resolved and issues that arose in the evolution of TEA. We adopted a three-stage approach (readiness to adopt technology, application of assessment technology, and evaluation/dissemination). The application stage adopted an assessment ‘lifecycle’ approach and targeted five key foci: (1) Advancing authenticity of assessment, (2) Engaging learners with assessment, (3) Enhancing design and scheduling, (4) Optimising assessment delivery and recording learner achievement, and (5) Tracking learner progress and faculty activity and thereby supporting longitudinal learning and continuous assessment.

Improvement Plan on the Utilization of Mobile Applications with Technology-Enhanced Assessment (TEA) Tool

Improvement Plan on the Utilization of Mobile Applications with Technology-Enhanced Assessment (TEA) Tool

Cultivating research skills through scholarly digital storytelling

ABSTRACT A fundamental responsibility of higher education institutions, across disciplines, is to develop student research skills. The exponential growth of digital scholarship, however, challenges traditional definitions of research. There is a need for deeper understanding of the ways in which students can develop and implement academic research skills and digital research skills simultaneously. This qualitative study explored graduate student engagement in scholarly digital storytelling, a technology-enhanced assessment, across multiple disciplines at a research university in the United States. Findings suggest that students developed the essential research skills of autonomy and flexible thinking along with multimodal research skills throughout the semester-long process of creating scholarly digital stories. Implications for utilizing technology-enhanced assessments, such as scholarly digital storytelling, in higher education are discussed.

Crisis and Changes in Learning Behaviours: Technology-Enhanced Assessment in Language Learning Contexts

ABSTRACT The outbreak of COVID-19 has posed a sudden and unprecedented change to the educational system. This study investigated the impact of this sudden change on EFL learners’ perceptions of assessment in a technology-enhanced learning environment. Assessment dimensions such as affective factors, validity, reliability, practicality, security and pedagogy were explored through an online questionnaire administered among 345 Iranian EFL learners and a focus-group interview with 17 learners. In addition, through a quasi-experimental design, the study compared the learners’ academic achievements of online and in-person classrooms during the COVID-19 outbreak. The findings revealed that the flexibility of online assessment, immediate feedback and encouraging self-regulated learning are among the advantages and technical problems, technology dependence and technology literacy are among the challenges of technology-enhanced assessment. The findings indicated that while technology-enhanced assessment has a positive impact on learners’ performances, learners have various attitudes towards e-assessment that need to be addressed in future studies.

Developing student capacity to produce digital scholarship in the humanities

Despite the increased use of technology in higher education classrooms, we need a better understanding of pedagogical strategies that improve student ability to produce quality scholarly digital content in the humanities. This research was designed to examine student learning through scholarly digital storytelling, a technology-enhanced assessment. The researcher collected data during and after an interdisciplinary, graduate scholarly digital storytelling course, including student work, student reflections, and individual interviews, to examine experiences at key points throughout the learning process. The results indicate that this pedagogical approach, when carefully scaffolded alongside formative feedback and ongoing student support, can increase student capacity—including digital agency, problem-solving skills, and digital knowledge production skills—to produce scholarly digital work in the humanities. Students can also learn to understand the interplay between disciplinary learning and digital skills and the ways in which both are essential for scholarly communication within and beyond the classroom.

Authentic learning across disciplines and borders with scholarly digital storytelling

Scholarly digital storytelling combines academic research and digital skills to communicate scholarly work within and beyond the classroom. This article presents three case studies that demonstrate efforts to integrate scholarly digital storytelling, a technology-enhanced assessment, across disciplines, geographic locations, and teaching contexts. The case studies originate in the United States, Northern Ireland [UK], and Norway, and represent learning across multiple disciplines, including history, higher education, geography, and biology. This article explores the potential for scholarly digital storytelling, when carefully planned, scaffolded, and implemented, to engage students in authentic learning, teaching students to think deeply and creatively about disciplinary content while creating sharable digital products.

A Systematic Review of Technology-enhanced Assessment of Monolingual and Bilingual Written Language Comprehension

A Systematic Review of Technology-enhanced Assessment of Monolingual and Bilingual Written Language Comprehension

Technology-enhanced formative assessment in higher education: A voice from Indonesian EFL teachers

The integration of information communication and technology in language teaching has been intensely examined in literature works. However, the study which explicitly investigates the advancement of technology for language assessment in higher education level is still underexplored, particularly in a formative assessment area. To fill that gap, the present study aims at investigating the perceptions of EFL teachers towards technology-enhanced formative assessment and how they implement it in the classroom. This study employs a qualitative case study approach. Of three participants from different universities were voluntarily participated in a semi-structured interview. The interview seeks to investigate the perceptions of teachers towards technology-enhanced assessment, the roles of technology in formative assessment, and the impact of technology on language learning. The findings reveal that EFL teachers view formative assessment as well as technology in a positive perception. They admit that technology is a practical tool that has multiple roles and it is deemed as useful and meaningful platform for assessing students. Finally, technology-enhanced formative assessment gives an impact on students� performance, particularly on language accuracy. The further implications are discussed in this paper.�

Acceptability of the e-authentication in higher education studies: views of students with special educational needs and disabilities

Trust-based e-assessment systems are increasingly important in the digital age for both academic institutions and students, including students with special educational needs and disabilities (SEND). Recent literature indicates a growing number of studies about e-authentication and authorship verification for quality assurance with more flexible modes of assessment. Yet understanding the acceptability of e-authentication systems among SEND students is underexplored. This study examines SEND students’ views about the use of e-authentication systems, including perceived advantages and disadvantages of new technology-enhanced assessment. This study aims to shed light on this area by examining the attitudes of 267 SEND students who used, or were aware of, an authentication system known as adaptive trust-based e-assessment system for learning (TeSLA). The results suggest a broadly positive acceptability of these e-authentication technologies by SEND students. In the view of these students, the key advantages are the ability of proving the originality of their work, and trust-based e-assessment results; the key disadvantages are the possibility that the technology might not work or present wrong outputs in terms of cheating.

El futuro de la evaluación en la educación superior

El reto pendiente de la evaluación en educación superior, aunque también en otros niveles, sigue siendo su vinculación efectiva con el aprendizaje de los estudiantes. A través de la evaluación se puede y debe conseguir un aprendizaje estratégico del estudiantado. Esta aportación surge a partir del intento de dar respuesta al interrogante sobre cuál sería el futuro de la evaluación en la educación superior, presentándose la misma bajo la forma de un texto elaborado de forma colaborativa entre todos los autores que la firman. Se ofrece así una reflexión conjunta de diversos autores de diferentes contextos y regiones sobre tres aspectos esenciales. En primer lugar, se resalta la necesidad de reflexión y un cambio en la evaluación basado en las tendencias actuales que están demostrando su oportunidad y validez. Un segundo aspecto se centra en el valor de la tecnología en los cambios que se están produciendo en la evaluación, pero siempre que se adapte a sus principios y, por lo tanto, no implique una vuelta al siglo pasado bajo el dominio de modelos hoy ampliamente superados. Interrelacionar evaluación y aprendizaje junto a la utilización de tecnologías que mejoran la evaluación implica, finalmente, rediseñar las prácticas evaluativas incorporando propuestas desde los ámbitos de la justicia social y la evaluación sostenible, el diseño de tareas de evaluación auténticas, propiciar retroalimentación de los procesos y fomentar la participación del estudiantado. En definitiva, colaborar en la formación del juicio evaluativo de los estudiantes para conseguir ciudadanos libres, socialmente responsables y justos.

Rethinking Assessment Concepts in Dental Education.

Introduction Dental education involves teaching and assessing the acquisition of verifiable domains that require superior psychomotor, communication, and cognitive skills. Evolving technologies and methods of assessment could enhance student's learning environment and improve tutor assessment experience. The aim of this study was to review the current body of research and evaluate the effectiveness of various methods of assessments in improving learning and performance in preclinical and clinical dental practice. Materials and Methods A search strategy was implemented using electronic search in major databases. The following key terms, clinical skills, preclinical, dental students, and assessment, were included in the search. Two reviewers independently screened all the articles retrieved following very specific inclusion criteria. Results The initial search generated 5371 articles and 24 articles were selected for review and data extraction. Cohen's kappa coefficient was used to measure interrater agreement and a score of 94.7% was obtained. Conclusion Preclinical assessment is an effective tool for promoting skills transfer to clinical phase. Early psychomotor skills assessment is valuable. It allows early intervention in the learning process and assists in effective utilization of learning resources. Technology-enhanced assessment systems allow better patient simulation, enhance learning and self-assessment experiences, and improve performance evaluation. However, these systems serve as an adjunct to conventional assessment methods. Further research should aim at calibrating and integrating these systems to optimize students learning and performance.

An Initial Study of Practicing Psychologists' Views of the Utility of Ecological Momentary Assessment for Difficult Psychotherapy Cases.

Ecological momentary assessment (EMA) is a burgeoning area of research, and several clinical applications of the resulting data have been identified by researchers, suggesting potential benefit to psychotherapy practice. However, practitioners often do not use traditional empirically-supported tools for diagnosis and outcome monitoring (e.g., validated interview measures and questionnaires). Thus, it is not clear how readily practitioners will take up newer technology-enhanced assessment methods, despite current enthusiasm among researchers. The current study aimed to explore the perceived usefulness of EMA-based tools for clinical assessment and outcome monitoring of difficult psychotherapy cases, as well as to identify correlates of attitudes about the usefulness of these tools. Clinical psychologists in active therapy practice with adults (n = 375) completed an internet survey including the Attitudes toward Standardized Assessment scale and the Attitudes toward Standardized Assessment Scales-Monitoring and Feedback. Respondents characterized their current diagnostic and outcome monitoring practices and rated how helpful they would find several assessment and outcome monitoring resources for a difficult case, including both traditional instruments and EMA-based methods. EMA-based tools had lower perceived usefulness than existing instruments. Attitudes toward standardized assessment and outcome monitoring predicted the perceived utility of these methods, as did several professional variables. Practicing psychologists may not adopt EMA for clinical assessment more readily than traditional assessment tools. Recommendations for facilitating the uptake of new technologies by psychotherapists are offered.