Marking Rubrics Research Articles

Exploring the engagement of students who identify as Black, Indigenous (and/or) People of Colour (BIPoC) with Learning Development practitioners: reviewing the value, impact, and recommendations for future practice

The need for universities to effectively support students identifying as Black, Indigenous, and/or People of Colour (BIPoC) remains a pressing element of strategies to close awarding gaps. Within overall support packages, the contribution of Learning Developers merits investigation, since these staff are often responsible for nurturing growth in students’ academic abilities. Through this small-scale, exploratory, qualitative study, we sought to better understand how Learning Developers can contribute to narrowing awarding gaps by exploring the benefits of and barriers to students engaging with Learning Development (LD). These benefits and barriers were elicited in interviews with twelve BIPoC students within one school in the Humanities and Social Sciences faculty in a UK university. The study explored their experiences of the value of LD interactions and how it might be enhanced. Aspects deemed valuable emerged broadly as anticipated, centring around how Learning Developers assisted students to learn academic practices such as: decoding assessment instructions and marking rubrics, referencing, breaking down tasks, and feedback uptake. More specifically relevant to BIPoC students, however, was the desirability of a flexible approach to the number and length of LD tutorials, enabling them to build relationships, understand guidance, and ask questions. Secondly, they mentioned the need to see ethnic diversity represented in the LD staff base and amongst student champions for LD. These factors were noted as good practice when evident, but open to further improvement. We conclude that whilst effective foundations were in place, the task of supporting BIPoC students remains one for ongoing reflection and action.

"Please may I have some more?" – Assessing Student and Instructor Intersubjective Agreement on Part Marks in Engineering Mathematics

Driven by student requests for more marks (some prefer ‘partial credit’), this paper asks if there is evidence that students adapt to the marking rubrics encountered in universities. Through a weekly handout in a first-year engineering mathematics class, we explore the progress students make in their understanding of how instructors grade. How does student expectation of an unknown grading rubric shift with the accumulation of experience in a course? Through our exploration, we reveal unfortunate findings about how well students adapt, a lot of unanswered questions, a few sources of good news, and promising directions for the research to pursue.

A Retrospective Cohort Analysis Comparing Analytic and Holistic Marking Rubrics in Medical Research Education.

The use of analytic rubrics in assessing and grading students' performance has become more prominent among instructors due to its reliability and validity in ensuring consistency in student evaluation. However, there is limited evidence demonstrating the consistency of examiner judgments between analytic marking rubrics and holistic marking rubrics. Therefore, we aimed to compare the consistency of marks given using holistic marking methods and analytic rubrics at an Australian university by analyzing the mean mark differences and number of adjudications between two rubric types as well as the inter-rater reliability between two assessors. We analyzed all scores for project manuscripts between 2016 and 2021 for Honours medical students. We compared the mean mark differences graded using the Kruskal-Wallis test and Welch t-test. We used chi-squared tests to compare the frequency of adjudications for each rubric type. We assessed interrater reliability by comparing the marks between the two examiners utilizing Pearson correlation. We found that analytic rubrics have lower mean mark differences and fewer adjudicators are required. We showed a strong positive association between the consistency of marks given and the use of analytic rubrics when compared to holistic marking. Pearson correlation showed a low but stronger correlation between marks awarded by the two assessors when analytic rubrics were used (r = 0.36), compared to holistic marking rubrics (r = 0.24). Our findings suggest that the use of analytic rubrics may increase the consistency and reliability between two independent examiners in marking medical students' work.

Ted's Teas: A Two‐Part Accounting and Audit “Crossover Case”*

AbstractThis fictional case is based on a real specialty tea company in Nova Scotia. Ted's Teas is a two‐part crossover case that illustrates the integration of financial accounting and audit learning outcomes applied to real‐world scenarios. In Part 1, students assume the role of an internal accountant and apply knowledge of the Chartered Professional Accountants (CPA) of Canada's CPA Way to identify how to treat leases, changes in policy, and estimates as part of accounting knowledge under both Accounting Standards for Private Enterprises and IFRS frameworks. In Part 2, students “cross over” and change roles, so they are now external auditors for Ted's Teas, tasked to provide an analysis of risk of material misstatement, recommend an audit approach, and develop substantive procedures. This two‐part case presents opportunities for students to demonstrate technical competence in multiple areas, either separately in two different courses or combined in one “capstone” or case course. The case and teaching notes, including the marking rubrics, were adapted from principles used to train and evaluate CPA Professional Education Program candidates, tailored to an appropriate level for undergraduate and graduate learners.

Analyzing Items Used in Developing Descriptive Essay Marking Rubrics: Focussing on Vocabulary Use

Driven by the pathological situation caused by this COVID-19 pandemic, this paper discusses the human ideal as argued by Nietzsche in his concept of the Übermensch (Superman). It also exposes the Filipino’s yearning to live the human ideal through “magpakatao” (to be human). Using textual analysis, I attempt to see the similarities and the differences between Nietzsche’s Übermensch and the Filipinos’ “ magpakatao” in order to come up with a synthesized and better concept of human ideal. To be Superman, Nietzsche encourages one to live life on earth to the fullest by rejecting inter alia, Christianity. Opposed to Nietzsche, Filipinos need to surrender themselves to Bathala (God) which at times hinder them live this material world to the fullest in Nietzsche’s sense. However, this paper concludes the relevance of Nietzsche’s philosophy to reconstruct the Filipino notion of “magpakatao.” With this reconstructed notion of human perfectabillity, I believe Filipinos and people all over the world could overcome any challenges that will come in their lives and still live and authentic and meaningful life amid this pandemic.

First-year writing in the humanities and social sciences: Requirements, expectations, and perceived weaknesses

This study aimed to explore writing requirements and lecturer expectations in first-year humanities and social science courses at a New Zealand university. Document analysis and lecturer interviews were used to collect data. The document analysis included writing assignment types, rhetorical functions of writing tasks, and descriptors in marking rubrics. Eight lecturers were interviewed to discuss their expectations of academic writing and perceptions of students’ writing problems. It was found that writing tasks require students to construct evidence-based scholarly arguments grounded on concepts from course readings. Students are expected to show their understanding and critical engagement with the readings, select ideas to strengthen their arguments and evaluate authors’ arguments. Counter-arguments are not required, but showing an awareness of different perspectives is important. Summarization is important for writing, but it is not often assigned as a stand-alone task in this context. These are the areas which challenge students, according to the lecturers. The findings imply that reading and writing instruction should be integrated, and rigid preformulated forms of writing should not be taught, particularly in high-level writing classes. Students need to be prepared to evaluate sources, integrate source ideas into their own essays, and construct scholarly arguments. Implications are discussed for teaching and assessment of writing as well as assessment of integrated reading and writing.

Instructors' Perceptions of Their Interaction With Students in Online Teaching and Learning

The study examined how 22 instructors (lecturers and tutors) perceived their roles in online teaching in a first-level course in open, distance and e-learning. The Community of Inquiry framework was used for the study. The research employed a case study involving questionnaires and document analysis. The findings revealed that the tutors saw their role as assisting students with assessments, online navigation, using ICT, motivation and time management. Lecturers perceived their roles as designing assignments and marking rubrics, updating course content, undertaking quality assurance and maintaining sound communication. Tutors spent between 16 and 37 hours and lecturers roughly 55 hours a week teaching the course. The study recommends effective communication, alternative automated assessments and lower student numbers per online class to decrease instructors’ workload and allow them to spend quality time interacting with their students. The findings will help higher education institutions with similar contexts to plan online courses with high student numbers.

An empirically based practical learning progression for generalisation, an essential element of algebraic reasoning

Generalisation is a key feature of learning algebra, requiring all four proficiency strands of the Australian Curriculum: Mathematics (AC:M): Understanding, Fluency, Problem Solving and Reasoning. From a review of the literature, we propose a learning progression for algebraic generalisation consisting of five levels. Our learning progression is then elaborated and validated by reference to a large range of assessment tasks acquired from a previous project Reframing Mathematical Futures II (RMFII). In the RMFII project, Rasch modelling of the responses of over 5000 high school students (Years 7–10) to algebra tasks led to the development of a Learning Progression for Algebraic Reasoning (LPAR). Our learning progression in generalisation is more specific than the LPAR, more coherent regarding algebraic generalisation, and enabling teachers to locate students’ performances within the progression and to target their teaching. In addition, a selection of appropriate teaching resources and marking rubrics used in the RMFII project is provided for each level of the learning progression.

Digital media assignments in undergraduate science education: an evidence-based approach

Digital media assignments empower students to become co-creators of knowledge rather than passive consumers of content. The Internet explosion and the affordability of digital technologies and devices such as smartphones, tablets and action cameras have created opportunities to use digital media in the classroom. This article aims to present an evidence-based approach to help educators to design, implement and evaluate digital media assignments in the classroom. For this purpose, four theoretical models were tested to inform the design of digital media assignments in undergraduate science education. These models helped to identify the student training in digital media needed, develop effective marking rubrics, and inform the design, implementation and evaluation of digital media assessment tasks. Trials were conducted in Spring 2016 (n = 458) and Autumn 2017 (n = 1329), respectively. Data collection used a mixed-methods approach, including a qualitative survey, open-ended questions, group contribution data and marks attained. Data analysis showed positive outcomes of the systematic implementation of digital media assignments. In conclusion, students enjoyed the support they received, being creative, working in groups and learning with digital media. To date, this intervention is one of the most comprehensive and practical approaches to digital media assignments in the classroom, which has been undertaken.

MATLAB-based project assessment in process modelling unit: A case study from Swinburne University of Technology Sarawak Campus

MATLAB has become one of the most widely used mathematical simulation software for process modelling and simulation globally. Integrating MATLAB learning into third year undergraduate Process Modelling unit nonetheless can be quite challenging given the students’ lack of prior programming knowledge and skills. Following a “Constructive Alignment” concept, a group-based MATLAB Project assessment has been developed here that shifts the focus from ‘what the teacher does’ to ‘what the students do’ to encourage them actively and independently learn MATLAB as they do the project. The students’ learning achievements in the Project are assessed through their group reports and presentations. Detailed marking rubrics are introduced for the reports evaluation while the individual group members’ learning achievements can be differentiated during the presentations.The observed trend and comments in the 2017–2019 student feedback surveys on the unit indicate that the implementation of the MATLAB Project has enhanced the students’ learning of the mathematical modelling principles and theories and that the self-directed or discovery learning methods adopted in this Project has encouraged them to discover process modelling for themselves via experimentation and exploration – relative to before the Project was introduced. This work also demonstrates the potential to use the students’ unit learning outcome (ULO) attainment data as a tool for continuous quality improvement, i.e., to determine what aspects should be modified or improved in the next year unit delivery to address the changes observed in the current year ULO attainment with respect to the previous years’ ULOs attainments.

Learning permeability and fluidisation concepts via open-ended laboratory experiments

This paper describes the implementation of open-ended laboratory for learning permeability and fluidisation concepts in Multiphase Processes unit and its impact on the students’ learning and understanding. The students were assigned specific objectives and were asked to devise and carry out their own experiments to evaluate the characteristics of flow through a bed of particles, to analyse the results of the experiments, and to competently report such results. The students’ performance was evaluated using the marking rubrics by assessing their ability to design their own procedure, to analyse and interpret the results of the experiments, and to work in a team. It is observed that all students achieved 78 % and above in both experimental run and laboratory report. One of the main factors that contribute to the good performance and low marks variation was the implementation of Level 2 open-ended laboratory, which comes with clear objectives to be fulfilled. The students were thus able to do better preparation prior to the laboratory session and did the necessary modifications to achieve the expectations more closely. The observed student feedback survey suggests that the application of the open-ended laboratory has improved the students’ learning of the permeability and fluidisation principles and theories and that the self-directed or discovery nature utilised in this open-ended laboratory has encouraged them to discover permeability and fluidisation concepts for themselves via experimentation and exploration. The feedback survey further reveals that the open-ended laboratory has provided them deeper understanding of permeability and fluidisation concepts and improved their attitude towards teamwork and effective communication skills. The students also favoured the open-ended laboratory over the traditional laboratory, particularly for the knowledge or skills gained, the interaction with the group members, and the trial-and-error nature of the experiments. Of the students who found the open-ended laboratory difficult, many responded being unaccustomed to designing their own experimental procedures. Nevertheless, one-third of them would recommend for more open-ended laboratories in other Chemical Engineering units. It is foreseeable that the implementation of fully open-ended laboratory (i.e., Level 3) would further increase the students’ learning and deeper understanding of permeability and fluidisation concepts.

Working with multiple representations: preservice teachers’ decision-making to produce a digital explanation

ABSTRACT A “digital explanation” is a science learning task where learners explain science content to non-expert others, in this case, the learners are primary preservice teachers [PST] in a science methods class. In the task, PST are assigned a prompt based on science content from the New South Wales K-6 syllabus and generate or source multiple representations to design and produce this stand-alone digital artefact. In this study, PST were interviewed about their decision making in producing the digital explanation, which offers insight into the design process and what makes for a successful product. Data include the digital explanations, interviews with nine PST, marking rubrics and rationale statements generated as part of the task. Thematic coding shows PST make design decisions for four principal reasons: content, engagement, clarity and unconscious selection. Decisions around content and engagement prevail and their decisions reveal keys to success in creating a digital explanation. We also note a tension between emphasis on engagement over content, which may reflect that the intended audience is young children and PST have a strong desire to gain and keep children’s interest.

Good and bad practices in rubrics: the perspectives of students and educators

A rubric is both an assessment and an evaluation tool that is widely adopted in education. However, little is known about good and bad practices in rubrics by either students or educators in nursing education, as well as in Asian higher education in general. The aim of this qualitative study was to explore the views of students and educators on good and bad practices in rubrics. A total of 82 participants, 78 nursing students and 4 educators, were involved in the study. Fourteen focus group interviews were conducted (13 interviews for students and 1 for educators), and six themes discerned. Good practices were categorised into: (1) standardisation of evaluation method, (2) objectiveness of evaluation, (3) guidelines for students’ work, and (4) transparency of evaluation. Bad practices in rubrics were: (5) vague descriptions in marking rubrics, and (6) failure to provide the ranges of marks for each grade. To conclude, good rubrics can ensure fair and consistent marking and lead to more objective assessments of student performance, thus promoting academic standards. This study can provide educators with insights into how to develop good rubrics and avoid bad ones.

P022: The revised METRIQ Score: an international, social-media based usability analysis of a quality evaluation instrument for medical education blogs

Introduction: Online medical education resources are widely used in emergency medicine (EM), but strategies to assess quality remain elusive. We previously derived the Medical Education Translational Resources: Impact and Quality (METRIQ) 8 instrument to evaluate quality in medical education blog posts. Methods: As part of a subsequent validation study (The METRIQ Blog Study), a mixed-methods usability analysis was performed to obtain user feedback on the quality assessment instrument in order to improve its clarity and reliability. Participants in the METRIQ Study were first asked to rate five blog posts using the METRIQ-8 Score. They then evaluated the METRIQ-8 instruments ease of use and likelihood of being recommended to others using a 7-point Likert scale and free text comments. Participants were also asked to flag and comment on items within the score that they felt were unclear. Global usability ratings were summarized using median scores or percent rated unclear. We used ANOVA to test associations between ease of use and demographic factors. A thematic analysis was performed on the comments. Results: 309 EM medical students, residents, and attendings completed the survey. Global ratings were generally very favorable (median 2 [IQR 2-3], with 7 being the lowest score) for ease of use and likelihood of recommendation, and did not vary by participants country of origin, frequency of blog use, or learner level. Participants stated that the score was structured, systematic, and straightforward. They found it useful for junior learners and for guiding blog creation. Four questions in the score (questions 2, 4, 5, and 7) were identified by 10% of subjects to be unclear. Thematic analysis of comments identified suggested four main themes for improving the score: adding clearer definitions with marking rubrics; shortening the 7-point scale; adding items evaluating blog post presentation and utility; and, rephrasing the wording of certain questions for clarity. Conclusion: A mixed methods usability analysis of the METRIQ-8 instrument for assessing blog quality was globally well received by EM medical students, residents, and attendings. Qualitative analyses revealed multiple areas to improve the instruments clarity and usability. The METRIQ score is a promising instrument for evaluating the quality of blogs; further development and testing is needed to improve its utility.

Practice makes perfect: a WID approach for Human Resource Management Honours students at a South African university

Little is known about how postgraduate students should be taught to write in their discipline. This research explores how a research preparation module supported Human Resource Management Honours students to write their research reports at a South African university. The module incorporated a ‘Writing in the Disciplines’ (WID) approach, because students, who worked in groups, wrote a series of developmental assignments marked according to rubrics that made the conventions of the report explicit. Many of the developmental assignments were rewritten as components of the research report. In order to determine if rewriting the assignments improved the students’ writing, the assignments from four groups were evaluated against the students’ rubrics. The redrafted assignments met more of the criteria in the marking rubrics. Since all group members spoke English as an additional language, this approach might benefit postgraduate students in similar contexts. The findings suggest that academic staff who want to offer developmental assignments prior to the submission of a larger research text should ensure that students are supported in two ways. Firstly, the design of the assignments should provide students with the opportunity to practice writing all the components of the larger piece of research writing. Secondly, supervisors and academic literacies practitioners should collaborate more effectively so that they can use what they learn from each other to better support students to write for their disciplines.
 How to cite this article:
 ARNOLD, Laura. Practice makes perfect: a WID approach for Human Resource Management Honours students at a South African university. Scholarship of Teaching and Learning in the South, v. 2, n. 1, p. 42-59, Apr. 2018. Available at: http://sotl-south-journal.net/?journal=sotls&page=article&op=view&path%5B%5D=25
 
 This work is licensed under the Creative Commons Attribution 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

A THREE-TIER EVALUATION RUBRIC FOR THE ASSESSMENT OF GROUP PROJECTS IN CHEMICAL ENGINEERING DESIGN COURSES

Abstract – Group projects are often key to engineering design courses since they simultaneously develop teamwork and communication skills in the context of solving difficult engineering problems. However, fair, consistent, and transparent grading of these projects are difficult to achieve, and the individual contribution of students can likewise be difficult to evaluate.
 Standardized marking rubrics are often used to increase the consistency and fairness of project evaluations; however, these frequently lack a systematic means for evaluating individual effort within group work. Rubrics also are difficult to employ when there are numerous possible solutions and where some solutions are more elegant or challenging when compared to others.
 To provide a consistent accounting of individual effort and the difficulty of a submitted group design solution, a three-tier marking rubric was developed. Comparing the project grades between two cohorts in the same course showed that there was a broader distribution of grades when using the three-tier marking scheme.

Using Debate to Understand How Unethical Decisions Can Be Made by Ethical Organizations

Debate can be a useful way to engage students with topics and stimulate discussion on what might appear to be a seemingly straightforward concept. This article describes a modified debate activity in which groups of six students debate preassigned topics in front of their classmates. The activity is designed to help students better understand how personal ethics and decision making can be influenced and challenged by various policies, procedures, and stakeholders. This article provides lesson planning suggestions, student handouts, and marking rubrics. This activity can be incorporated into any undergraduate or graduate level course that has content dealing with ethical decision making.

Collaborative Learning in Authentic Environment Apps to Promote Preschool Basic Scientific Process Skills

This study was carried out to evaluate the effectiveness of mobile learning app based on collaborative learning in authentic environment (known as AKSES) and its impact on the preschool students’ basic scientific process skills competency namely; observation, classification and communication. In addition, the study also investigated competency patterns of the students’ skills. This study was a quantitative study which used two sets of instruments which looked into AKSES application and marking rubrics of the students’ basic scientific process skills. Based on quasi-experimental study, the duration of the study was seven weeks and six learning activities were implemented. Two rural preschools were selected based on quota sample selection and 25 students were in each controlled and treatment groups. Analysis of the survey data was carried out in the form of descriptive and inferential analysis involving Mann-Whitney U test. The Mann-Whitney U test analysis found significant differences in competency patterns of basic scientific process skills in both groups. Meanwhile, the competency patterns of basic scientific process skills indicated positive increase among all preschool students after the utilization of AKSES application in learning activities based on collaborative learning strategy in an authentic environment.

Listenin’ Up: Re-imagining Ourselves through Stories of and from Country

Listenin’ Up: Re-imagining Ourselves through Stories of and from Country

Evaluation of Student Work Using a Letter Grading Scheme in Design and Calculus

This paper presents methods, pros and consof using a letter grading system versus a percentagegrading system, in engineering course componentevaluations. In making evaluation criteria acrossdeliverables qualitatively similar, letter grading showsperformance equivalency across courses and subjectareas, as well as departments, faculties and universities.What is worth an ‘A’ is not always a trivial discussion.But it is an easier discussion than what is worth 83%versus 85%. How to letter grade various types ofdeliverables in a valid and equivalent fashion can still bechallenging. But with thoughtful marking rubrics, manydeliverables can be evaluated using letter grades. Thesecan be combined to produce valid final letter grades.Overall, there are advantages to taking such an approachto evaluation, and these are discussed in the paper.