Learning Assessment Research Articles

Improving assessment for work-based learning and degree apprenticeships in business through a co-designed peer review process

Involving students in peer review has substantial benefits for students’ learning and development and, more pragmatically, for staff workload. This paper reports on a student-led effort to co-design a peer review formative assessment task. The assessment is part of a personal development module within a higher education degree apprenticeship programme delivered via blended mode by a business school in South-East England. This case study on student-staff partnership offers a description of the process and a reflection on the impacts from both the student and staff perspective. It suggests that student co-design and peer review activities complement work-based learning, effectively bringing the prior experiences and academic skills of mature/professional students into the classroom.

Trend Analysis of Fluency-Related Research Articles: Using Data Mining

This research seeks to identify recent research trends in analysing fluency in English language studies since 2010. The importance of studying corpora that compile registered research in linguistics is growing; thus, it is time to consider the significance of data mining analysis on the corpora. Considering this emerging research topic, this analysis intends to generate and compare word clouds and word metric charts from speaking and writing abstract corpora in linguistics research centred on the keyword ‘Fluency’. This comparison is minimally addressed in extant research; hence, this investigation aims to fill this gap. The corpus contains 50 speaking and 15 writing abstracts from linguistics journals. To create the word cloud, AntConc4.1.4 software was used to analyse the corpora, while TF-IDF and matrix analysis were conducted using R. As a result, in spoken English, ‘fluency’ studies were largely related to 'teaching' and tended to be dominated by studies focusing on ‘fluency’ alone. In contrast, writing studies were mostly related to learner proficiency and assessment, and many studies analysed the relationship between linguistic abilities by expanding the scope of ‘fluency’ to include ‘accuracy’ and ‘complexity’. This study will benefit researchers in deciding on topics as it provides research directions and trends in analysing fluency in the two interconnected but differently expressed fields (i.e., speaking and writing).

Balancing Honest Assessment and Compassion for Learners Experiencing Burnout: A Workshop and Feedback Tool for Clinical Teachers.

Burnout in medical trainees challenges their work effectiveness and impedes their ability to learn. Teachers in the clinical learning environment (CLE) are well situated to identify burnout and are often responsible for learner assessment. We developed a workshop to improve clinical teachers' identification and understanding of learner burnout while empowering them to provide constructive feedback and support. Building on best-practice feedback techniques and utilizing the Maslach Burnout Inventory (MBI) as a framework, we designed the GetINBurnOUT method to provide feedback and support for learners experiencing burnout. Applying Kern's six-step approach to curriculum development, we created and implemented a workshop for clinical teachers centered on advanced burnout knowledge, application of the MBI to the CLE, and use of the GetINBurnOUT feedback method. Kolb's experiential learning theory informed workshop activities such as group discussion, case practice, and self-reflection. Participants completed surveys immediately after the workshop to assess planned behavior. We delivered the workshop eight times at local, regional, and national faculty development programs/conferences to over 188 participants. Participants rated the workshop favorably, with average scores of 4.5-4.8 out of 5 across all domains and program objectives; all participants planned to make a change to their practice. Positive comments emphasized the topic's importance and the GetINBurnOUT tool's practicality. This workshop can enhance clinical teachers' understanding of burnout and provide them with the tools to address it in the CLE. The GetINBurnOUT method offers a practical approach for providing honest assessments while supporting learners in the CLE.

EPortfolio-based Professional Development: Insights from a Southeast Asian Online Educators’ Community of Practice

This study examines the implementation and impacts of an ePortfolio-based professional learning initiative within a Southeast Asian online community of practice (CoP), part of the SEAMEO-AUS EduLink Project. Targeting the development of educators' competencies in ePortfolio pedagogy, the project engaged 541 educators from various Southeast Asian countries in two online self-paced courses and a series of four expert-led webinars, culminating in a final project webinar. The research utilised course analytics and post-webinar surveys to analyse participant engagement, satisfaction, and perceived learning outcomes. The findings indicate substantial engagement with the courses and positive reception of the webinar series. About 75% of participants completed the asynchronous courses, while 64% found the webinar content immediately applicable to their practice. The webinars, focusing on diverse aspects of ePortfolio use, successfully complement the online courses, with 91% of participants expressing intent to attend future events. This study underscores the effectiveness of combining asynchronous and synchronous learning modalities in professional development, particularly in the context of ePortfolios. It highlights the role of ePortfolios in promoting reflective practice, collaborative learning, and authentic assessment. Additionally, the project's success in fostering a supportive CoP model illustrates the potential of online communities in professional development, especially in regions with emerging ePortfolio practices. The paper concludes by emphasising the value of ePortfolio-based initiatives in enhancing educators' skills and fostering cross-cultural educational collaborations. It offers recommendations for educators and policymakers on implementing similar initiatives, underlining the need for sustainable and contextually sensitive approaches in professional development.

The SOLIO taxonomy: an integrated model for assessing student learning and interest

ABSTRACT Assessing student learning outcomes has been an integral and central component of schools and academic institutions, both as a measure of how well the school is performing and how well individual students are progressing. However, standardised tests might not be able to fully capture holistic aspects of learning such as self-regulatory competencies, motivation and interest. The Structure of Observed Learning Outcomes (SOLO) taxonomy has been used to examine the quality of learning outcomes. Based on findings from the current study in a STEM-based learning curriculum, a new model, the Structure of Observed Learning and Interest Outcomes (SOLIO) taxonomy, is proposed to capture student interest development as well as learning outcomes for a more holistic assessment of learning.

Program Implementation of the Alternative Learning System

Educational stakeholders can enhance the effectiveness and impact of the Alternative Learning System (ALS) programs in providing educational opportunities for marginalized and underserved populations. It is on this premise that this study aimed to evaluate the extent of the ALS program’s implementation in a large school division within Central Philippines for the 2021-2022 academic year. Data needed for this descriptive study was taken from 51 ALS teacher-respondents using a self-made survey questionnaire that passed the rigorous test of validity and reliability. The study’s focus on program implementation revealed that the Assessment of Learning component exhibited a very high extent of implementation, as evidenced by mean scores surpassing 4.5 across all assessed aspects. Similarly, Learning Resources demonstrated a high level of implementation, with the majority of mean scores exceeding 3.5. However, implementing the Learning Environment aspect was deemed moderate, signifying potential areas for enhancement. Regardless of age, both younger and older educators experienced robust implementation in the Assessment of Learning and Learning Resources aspects. Similarly, male and female teachers encountered high implementation levels in these dimensions. While family income did not significantly impact the Assessment of Learning and Learning Resources domains, it did influence the Learning Environment aspect. Specifically, higher-income individuals benefited from a more extensive implementation of the Learning Environment. The study’s outcomes have important implications for designing future capability-building plans and enhancing the educational experience within the ALS program. Keywords: Alternative Learning System, ALS Program Implementation, ALS Teachers, ALS Community Learning Center, ALS Learning Resources, ALS Assessments Negros Occidental

Difficulties Encountered by Mapeh Teachers in the Blended Learning Modality

Examining the challenges MAPEH teachers face in the blended learning environment aims to shed light on the difficulties they encounter. In this context, this study aimed to determine the difficulties experienced by MAPEH teachers in the blended learning modality (BLM) in three schools of District VI Division of Bacolod City during the school year 2021-2022. Data for this descriptive study was collected from 101 teachers using a researcher-made instrument that has gone through rigorous tests of validity and reliability. The ensuing analysis showed MAPEH teachers encountered high-level challenges in delivering instruction and utilizing technology in blended learning, with a moderate level of difficulty in assessing learning. Younger teachers with shorter teaching experience, particularly those assigned to primary grade levels, faced moderate challenges in technology literacy and application and assessment, whereas older teachers experienced higher difficulty levels. Significant differences were found in the challenges MAPEH teachers face across age groups in technology literacy, application, and assessment in blended learning. The findings of this paper call for support through seminars and training for the teachers, especially in delivering instruction and effectively utilizing technology in blended learning. Keywords: MAPEH teachers, empirical-based approach, difficulties in blended learning modalities, Bacolod City, Philippines

Exploring the perceptions of literacy in assessment for learning among high school English teachers

In the field of English language education, the crucial role of assessment for learning (AfL) requires teachers to possess robust assessment literacy. This study explores AfL literacy among high school English teachers in Yogyakarta, Indonesia by utilizing Alonzo's validated AfL survey that establishes a comprehensive six-factor model, delineating teachers as assessors, pedagogists, student partners, motivators, learners, and stakeholder partners. Exploiting confirmatory factor analysis and examining demographic variations, this quantitative research invited 202 English teachers in the Special Region of Yogyakarta, Indonesia selected purposively based on the geographical service area. Data were collected through an online questionnaire adopting Alonzo's 42-questions AfL and were analyzed quantitatively via Confirmatory Factor Analysis (CFA) with four indices, namely comparative fit index (CFI), Tucker-Lewis index (TLI), root mean square error of approximation (RMSEA), and standard root mean square residual (SRMR).The findings substantiate the efficacy of the six-factor AfL model, underscoring educators' roles extending beyond traditional frameworks. The investigation also introduces a tool featuring detailed performance descriptors, addressing deficiencies, and harmonizing with AfL principles. It deduces that heightened foundational comprehension among English educators cultivates enhancements in AfL literacy and propels the refinement of professional evaluative competencies, thereby enriching the nuanced discourse surrounding AfL within language pedagogy. While the study's scope is confined to a specific geographical area and a limited cohort of participating instructors, it significantly enriches our comprehension of AfL literacy among English pedagogies. This research, therefore, provides a foundation for professional growth initiatives and facilitates enhancements in pedagogical approaches and academic achievement.

Evaluation Proposal for School Teaching and Learning

Objective: This work aims to present a proposal for an evaluation method, entitled: Cycle of Autonomous Learning Evaluation (Caaa). Theoretical Framework: The use of assessment instruments in schools as a genuine assessment and the low level of student learning in Brazil exposes the emerging need to rethink the assessment practices that are carried out, since in order to raise their learning to a higher and better quality, it is essential to qualify the means that give access to it. Method: In order to construct this study, we immersed ourselves in educational theory relating to learning assessment in the school context and active teaching practices, as well as in the documents that underpin Brazilian education and what they guide in terms of learning assessment. Results and Discussion: The studies carried out resulted in the development of the assessment method: Cycle of Autonomous Assessment of Learning (Caaa). Research Implications: The practical and theoretical implications of this research are discussed, providing insights into how the Cycle of Autonomous Assessment for Learning (Caaa) can be applied or influence practices in the field of education. Originality/Value: This work contributes to the development of the praxis of school learning assessment in Brazil.

An In-Depth Evaluation of Educational Burst Games in Relation to Learner Proficiency

Game-based learning assessments rely on educational data mining approaches such as stealth assessments and quasi mixed methods that help gather data on student learning proficiency. Rarely do we see approaches where student proficiency in learning is woven into the game’s design. Educational burst games (EBGs) represent a new approach to improving learning proficiency by designing fast-paced, short, repetitive, and skill-based games. They have the potential to be effective learning interventions both during instruction in the classroom and during after-school activities such as assignments and homework. Over five years, we have developed two EBGs aimed at improving linear algebra concepts among undergraduate students. In this study, we provide the results of an in-depth evaluation of the two EBGs developed with 45 participants that represent our target population. We discuss the role of EBGs and their design constructs, such as pace and repetition, the effect of the format (2D vs. 3D), the complexity of the levels, and the influence of prior knowledge on the learning outcomes.

Examining the influence of list composition on the mnemonic benefit of errorful generation

ABSTRACT Despite literature showing that errorful generation with corrective feedback enhances retention better than mere studying, it is unclear if this benefit depends on the composition of the learning list (pure error generation/read versus mixed). Here, we investigated whether the mnemonic advantage and metamnemonic evaluation of errorful generation generalise beyond mixed-list designs. Experiment 1 used a free-recall test, while Experiments 2 and 3 used a cued-recall test, with Experiment 3 also including a judgment of learning (JOL) assessment. Only when memory was tested via free recall did the benefit of errorful generation depend on experimental design, with the effect being most robust in mixed lists. Replicating past research, we too found that despite a clear mnemonic benefit for error generation in cued-recall tests, participants predicted better memory following read-only trials, and that this effect was not contingent on list composition. At the practical level, these findings demonstrate instances in which errorful generation is beneficial for memory and learning. At the theoretical level, the results fit nicely within the item-order framework in accounting for commonly observed design effects in free recall.

An advanced machine learning and meta-heuristic approach-based heat transfer assessment of conformal cooling channel in injection molding process

An advanced machine learning and meta-heuristic approach-based heat transfer assessment of conformal cooling channel in injection molding process

Enhancing children’s learning experience through alphabet ar-game: a usability and acceptability assessment of augmented reality game-based learning

Enhancing children’s learning experience through alphabet ar-game: a usability and acceptability assessment of augmented reality game-based learning

A Framework for the Communication Capabilities of scientists

ABSTRACT The ability to communicate science effectively is a key attribute for science graduates, and considerable work has been done to establish the techniques and skills associated with effective communication. What is less clear however, is the learning progression students traverse as they develop proficiency in science communication. In this study we evaluated aspects of university-level student science communication capabilities pre- and post- explicit teaching of core science communication capabilities. We used this data to identify the characteristics of early science communication learners and synthesise a robust learning progression Framework for Communication Capabilities (FCC). The FCC was created to support educators in identifying the characteristics of their learners to facilitate the constructive alignment of learning objectives, curriculum, and assessment rubrics for learners of science communication through early phases of their development.

Swedish L2 teacher cognitions of the initial assessment of students' L1 literacy resources

In this paper, we explore second language (L2) teachers’ cognitions regarding the initial assessment of adult learner literacy in their strongest language. Instruction can thus be adapted according to what is stipulated in the curriculum and syllabus. The literacy assessment is conducted in the student’s strongest language with assistance from an interpreter, translated decoding and reading comprehension tasks and concerns assessment of learners with little or no prior education. Questionnaires and interviews with L2 teachers reveal great variation based on four categories of teacher knowledge, namely, Subject matter knowledge, Knowledge of students, Pedagogical knowledge, and Knowledge of educational contexts. These four categories were partly intertwined with each other and both a resource and a deficit perspective emerged. The teachers’ cognitions regarding the concept of literacy, the value of initial L1 literacy assessment and students’ prior literacy and multilingual resources, as well as its potential for instruction planning were diverse. This may have significant implications for what emerges in initial assessments of a student's L1 literacy. In turn, this affects the validity of the assessment and, ultimately, the quality of education.

Implementing Differentiated Instruction in Emancipated Curriculum: Narrative Review

The paper aims to provide a detailed description of the implementation of differentiated instruction in an emancipated curriculum. The purpose of this review was to provide a comprehensive summary of the current literature on a specific topic. Implementing differentiated teaching can be analyzed from four perspectives: learning material, instructional process, assessment, and learning environment. Through the implementation of differentiated instruction, teachers can actively involve students in the process of teaching and learning, hence enhancing students' academic performance. Therefore, diversified learning may accommodate the diverse needs, cognitive preferences, and abilities of each student in a classroom environment. This approach not only promotes equity but also enhances student engagement and achievement. Professional educators are recommended to employ differentiated instruction as a method to actively include pupils in the learning process.

Exploring Jalisco's water quality: A comprehensive web tool for limnological and phytoplankton data

AbstractThis study presents a comprehensive dataset of hydrological information gathered from five key eastern basins in Jalisco, Mexico. The dataset encompasses approximately 50 limnological variables and phytoplankton counts specifically for one of these basins. Water‐quality data were collected by the State Water Commission of Jalisco, adhering to the methods outlined in the Official Mexican Norm ‘NOM‐127’. Monthly samplings were conducted to assess environmental variables such as pH, temperature, oxygen, nutrients and heavy metals. Monitoring has been ongoing for three basins since 2009, while the remaining two basins have been monitored since 2015 and 2020. Phytoplankton data were obtained from monthly samples taken by the University of Guadalajara between 2014 and 2019 in Lake Cajititlán. The original data were cleaned and organized using tidy data principles, with codes accessible on GitHub. To facilitate data exploration and visualization, we developed a user‐friendly web application with the Shiny package in R. This application enables users to explore the dataset through summary statistics tables, time series plots and phytoplankton community analysis. The dataset is accessible on Zenodo. The presented data hold significance for environmental and water‐quality assessment and applications in machine learning, neural network models, community ecology and broader environmental research. Notably, the raw data, publicly accessible from the State Water Commission of Jalisco, have been previously utilized for these purposes. This dataset offers value due to its diverse limnological and phytoplankton variables, an extended time frame of availability, a curated and streamlined accessibility process and the inclusion of a web application for intuitive exploration and visualization.

Analysing Game Data to Evaluate Children’s Learning Progress with the EDURINO App

Game data offers a unique way to examine children’s learning progress in a gameplay context. Despite its relevance in evaluating the efficacy of game-based learning, this area is less researched than engagement and usability study. This paper presents a quantitative investigation of EDURINO, an educational gaming app targeting children aged 4-8. We implemented learning metrics to track children’s performance in EDURINO games. Specifically, the measurements included standardised variables from educational and experimental psychology, such as error count and duration. Here we reported data from a total of 18 EDURINO minigames in Literacy and Maths. Game data was collected by the learning metrics tracking system. A total of 66,463 players’ data was analysed using generalised linear mixed models with the lm4 package in R. Overall, children have shown improvements while playing the EDURINO app. We examined the relationships between children’s game performance, player demographics and game-specific features and discovered different patterns. First, the significant main effects of age and gender indicated that EDURINO’s educational content was developmentally sensitive and age appropriate. Second, children made fewer errors and spent less time when they were in the later rounds within a game or have played the same game more than once. Third, higher difficulty level undermined game performance. Worth noting is that the effect of game-specific factors varied among different educational content, and such discrepancy could be attributed to the nature of game design. When presented with engaging and interactive content, children may have chosen to deliberately make mistakes for fun effects. These findings are discussed in relation to game-based assessment for learning in EdTech research and industry. We also considered the interpretation of learning progress in EDURINO’s gameplay context while highlighting the need for further efficacy research.

Machine Learning Assessment of Gestational Age in Accelerated Maturation, Delayed Maturation, Villous Edema, Chorangiosis, and Intrauterine Fetal Demise.

Assessment of placental villous maturation is among the most common tasks in perinatal pathology. However, the significance of abnormalities in morphology is unclear and interobserver variability is significant. To develop a machine learning model of placental maturation across the second and third trimesters and quantify the impact of different pathologist-diagnosed abnormalities of villous morphology. Digitize placental villous slides from more than 2500 placentas at 12.0 to 42.6 weeks. Build whole slide learning models to estimate obstetrician-determined gestational age for cases with appropriate maturation and normal morphology. Define the model output as "placental age" and compare it to the chronologic gestational age. Our model showed an r2 of 0.864 and mean absolute error of 1.62 weeks for placentas with appropriate maturation in the test set. Pathologist diagnosis of accelerated maturation was associated with a 2.56-week increase in placental age (±2.91 weeks, P < .001), while delayed maturation was associated with a 0.92-week decrease in placental age (±1.82 weeks, P < .001). Intrauterine fetal demise causes diverse changes to placental age, driven by the nature of the demise. We tested the impact of training a model, using all live births. The resulting r2 was 0.874 and mean absolute error was 1.73 weeks. Furthermore, by including cases with abnormal maturation in the training data, the effect size of accelerated maturation was blunted to only 0.56 ± 2.35 weeks (P < .001). We show that various abnormalities of villous maturation and morphology correlate with abnormalities in placental age. This "no pathologist" model could be useful in situations where pathologists are unavailable or the need for consistency outweighs the utility of expertise.

Game-based Learning Assessment: Quantifying Educational Efficacy of the EDURINO App

Assessing the potential and efficacy of educational apps is crucial for parents, educators and EdTech stakeholders. This paper presents evidence from two quantitative studies on EDURINO, a game-based learning app targeting children aged 4-8. It features seven subject areas and is designed to be used with an ergonomic pen that facilitates fine motor skill development and pen-holding gesture. In Study 1, parents (N = 45) used a validated index (Kolak et al., 2021) to evaluate the app’s educational potential across 10 features. In study 2, a larger cohort of parents (N = 235) reported on their children’s school readiness and fine motor skills. Specifically, we adapted the Brief Early Skills and Support Index (BESSI) to measure changes in behaviours and cognition. We also included items from the Child Development Inventory and Ages and Stages Questionnaire to assess fine motor skills. Study 1 results indicated high ratings for EDURINO’s meaningful learning and social interactions, but lower for adaptive learning. Parents’ evaluation of EDURINO did not differ by child gender, age, or subject area, but varied with parental education level (the higher the educational attainment, the lower the ratings). Study 2 found that there were statistically significant improvements in children’s behavioural and cognitive development as well as fine motor skills after playing EDURINO. Changes in those areas were independent of children’s age, gender, and subject area. Moreover, children’s improvements were positively correlated with the length of playing. However parental engagement level influenced and was negatively associated with children’s improvements in EDURINO targeted area. Overall, these studies provide evidence on EDURINO’s efficacy while highlighting the necessity for improvement with personalised and adaptive learning. The importance of transparency and collaboration between academia and industry, along with the influence of demographic factors are discussed.