Evaluation Rubrics Research Articles

Do Pre-Service Turkish Language Teachers Succeed In Developing Argumentative Writing Skills?

The purpose of the study is to determine pre-service Turkish language teachers' use of text structure elements and their awareness and experience with argumentative writing. The research was designed as a case study, which included 115 undergraduate students studying Turkish language teaching. The data of the study consisted of the participants' argumentative writings and interviews. As data collection tools, the researchers developed an "Evaluation Rubric of Argumentative Text Elements" by conducting validity and reliability tests and prepared a "Semi-structured Interview Form" to uncover the participants' experiences and opinions about argumentative writing. Quantitative data were evaluated using descriptive statistical techniques, and qualitative data were analyzed through thematic analysis. The quantitative results showed that the average success of the participants was 75.2% in topic element, 72.6% in claim element, 65.6% in the element of supporting a claim, 42.3% in counterclaim and backing for the counterclaim, 36.8% in grounds for rebuttals, 49.3% in conclusion element, and the overall average success was 56.54%. The qualitative results indicated that the participants' awareness of argumentative text structure was weak. Lastly, the participants' experiences with and opinions on argumentative writing were categorized to determine the factors that contributed to their success.

Vascular Surgery Integrated Resident Selection Criteria in the Step 1 Pass/Fail Era: A National Survey of Program Directors

Vascular surgery integrated residency (VSIR) programs are highly competitive; however, criteria for resident selection remain opaque and nonstandardized. The already unclear selection criteria will be further impacted by the impending transition of the United States Medical Licensing Examination (USMLE) step 1 from numeric scores to a binary pass/fail outcome. The purpose of this study is to investigate the historical and anticipated evaluation rubric of VSIR applicants.

Improving team dynamics for project based learning in pharmacy: A multimodal approach

Background and purposeStudents taking a research elective with project-based course components have shown aversion to group activities. We aimed to minimize group participation hesitancy and give students autonomy in choice of team formation approach in order to examine the effects of team formation approaches on successful team dynamics. Educational activity and settingLearners chose either a student self-selected (SS) or an instructor randomized (IR) team formation approach for two activities (a brief intervention role-play and a review of reviews research symposia presentation). Group development for the different team approaches was studied using the Tuckman model. Using this model, team dynamics was evaluated over five stages of group development: forming, norming, storming, performing, and adjourning. Student reports for each of the phases were evaluated using a project evaluation rubric. For the adjourning phase we used an open-ended survey embedded in the course learning management system. Free text answers from open-ended questions were analyzed for themes related to team dynamics concepts. FindingsStudents rated their satisfaction with team performance higher for SS than for IR teams. In terms of individual learning and satisfaction with individual's roles and tasks, they indicated greater satisfaction with the IR approach. SummaryTeam formation methods impacted group dynamics and individual attitudes with favorable team dynamics leading to better individual task and overall team performance. Higher team performance corresponds to higher grades for group projects and for courses with group projects, favorable team dynamics could impact students' evaluation of the course.

Design Sprint: Enhancing STEAM and engineering education through agile prototyping and testing ideas

Creating project-based learning experiences in the classroom where students learn in a team to solve complex problems and to develop creative and critical thinking is a challenge. Design Sprint (DS) is an agile methodology (implemented in 5 days) with the goal of creating innovative design based on user needs (User Experience). The objective of this work was to develop an Engineering Drawing classroom experience linked to the context of the current COVID-19 pandemic with the Design Sprint methodology. The experience had to involve the integration of theory and practice, the application of knowledge, the development of both hard and soft skills, and the empowerment of students to conduct research. 56 first-year students following three STEAM degrees at the University of A Coruña participated in this experience. The activities were designed for both face-to-face and remote learning. Microsoft Teams and Moodle were used for tutoring and for monitoring student progress. The Moodle Workshop tool was used for the evaluation of the prototypes that were developed and the projects were evaluated by video. The students defended their projects through a presentation in lightning talk format (Ignite). Evaluation rubrics were used following a triple approach: co-evaluation, hetero-evaluation and self-evaluation. The 3D design of the projects was developed with Autodesk software. A total of 18 projects were developed. Once the projects were completed, a survey was administered to evaluate the levels of student satisfaction. The survey results were very positive. The Design Sprint projects also showed positive effects on grades. The Design Sprint method has promoted an interactive learning environment. In addition to its simplicity, a further advantage of DS method is that all student dedication is planned. Students were therefore less likely to feel overloaded, all of which helps with better time management. The DS methodology is multipurpose, so it can be applied to various fields and subjects.

Evaluation of students' self-assessment performance in preclinical endodontic training by means of rubrics and a 3D printed model

Effective evaluation of students' self-assessment ability is crucial. This study was to develop a standardized 3D printed teaching model accompanied by structured scoring rubrics for preclinical endodontic training, to appraise students' self-assessment ability, and to evaluate their perceptions of the training system. The 3D printing model was designed to house a standardized central incisor. Forty-four undergraduate dental students were enrolled and their endodontic performance was self-assessed and also rated by a teaching assistant and a tutor using 3 structured rubrics including access cavity, mechanical preparation, and obturation assessments. In total, 21 rubric evaluation items of preclinical tasks were assessed, and the assessment results from the student, teaching assistant, and tutor were compared. Furthermore, questionnaires were used to evaluate students' experiences with the new system. The tutor, teaching assistant, and students all had similar scorings and high intraclass correlation coefficients (ICC) for mechanical preparation and obturation rubrics, while the students had underestimated scores on the access cavity rubric with a rather low ICC of 0.387, which also lead to the underestimated overall scoring (P<0.05). Among the 21 evaluation criteria, 9 items were rated statistically different (P<0.05), which denoted the major deficiencies of students' self-assessments. More than 80% of students rated satisfied for most of the questions regarding the new training system, except the tactile sensation, hardness, and radiopacity. The proposed standardized 3D printed model and structured scoring rubrics is feasible for preclinical endodontic training, and standardized evaluation of students' self-assessment ability.

Rubrics in evaluation

This research paper designs to show the quintessential importance of the rubrics in evaluation of the learners in the outcome based education. The paper attempts to communicate expectations of quality around a task. Rubrics is an evaluative criterion which can be used to measure expected levels of performance of the learners. Rubrics can make brilliant culmination in the traditional approach to assess learners. It is used to delineate consistent criteria for grading. Outcome based education is flourished with rubrics in evaluation system. Rubrics allows teachers and students alike to evaluate criteria which can be complex and subjective. Scoring rubrics can also provide a basis for self-evaluation, reflection and peer review. It aims at fair assessment, fostering understanding, motivating, indicating a way to proceed with subsequent learning. Rubrics can be classified as holistic, analytic or developmental. Holistic rubric integrates all aspects of the work into a single overall rating of the work. Learners can better be judged and revised by the teachers through rubrics in diverse kinds of evaluation. A rubric measure students work against real life criteria and is referred to as a form of authentic assessment.

Design, Development, and Evaluation of the Robobug Board Game: An Unplugged Approach to Computational Thinking

This paper presents the design, development, and evaluation of the Robobug board game, which was developed to foster computational thinking (CT) among primary school pupils in Malaysia, particularly those who encounter computer and Internet access issues. Utilizing Richey and Klein's design and development (DDR) methodologies, this research serves as a primer by first concentrating on needs analysis for the design, then converting it into the development of the board game. The focus of this study was on how the design aspects of Robobug might help children grasp CT principles and foster CT practices. The Robobug board game was initially assessed remotely via questionnaire by a panel of eight subject matter experts from Malaysian institutions. Using the questionnaire and evaluation rubrics, experts were asked to offer insights with the following objectives: (i) to evaluate the design features; (ii) to examine the computational concepts; and (iii) to examine the computational practices of the Robobug board game. The research found that the Robobug board game has significant potential for use as an unplugged tool and alternative technique for instructors to enhance CT abilities outside the typical classroom. Additionally, it can be utilized to incorporate collaborative and game-based learning into teachers' instructional strategies to boost student engagement and achievement. The Robobug board game's shortcomings and suggestions for improvement are discussed.

Tecnoetica: Inclusion and Assessment of the Ethical Competence in Engineering Final Degree Projects

Ethics is taking a relevant role in this 21st century, in most cases going hand in hand with scientific-technological advances and the social and cultural implications. However, to date, the evidence on which a university in the European Higher Education Area (EHEA) can rely to incorporate the ethical perspective in the educational curriculum successfully is still scarce. To increase the body of knowledge in engineering ethics, this article presents and evaluates the initiative called ’Tecnoetica’ implemented at the University of Deusto, Spain. This initiative proposes to include training in ethical principles, dilemmas, and issues from a holistic and participatory standpoint in which lecturers and students are involved. Because of the initiative, an evaluation rubric is presented as the main contribution from which other centers, universities and/or faculties can benefit.

Searching for equity-centered change agents: An investigation of family and community engagement in teacher evaluation frameworks

Federal legislation in the United States granted states freedom to alter formerly test-based teacher evaluation systems and situate family engagement as a key component in school improvement efforts. Concurrently, theorizing on family engagement has moved away from deficit characterizations and school-based involvement to an asset-based approach focused on equitable stakeholder collaborations committed to driving systemic change. Seeking to understand how states are characterizing exemplary teaching in terms of family and community engagement, the current study uses directed qualitative content analysis to examine 15 teacher evaluation rubrics or standards. The findings indicate that many evaluation systems were narrowly focused on the procedural aspects of teaching and teacher engagement with families was too often viewed merely as a mechanism to yield academic gains and meet legal requirements. The rubrics that explicitly stated that teachers should be culturally responsive provided a semblance of hope for pushing evaluative practices through the engagement barrier and into a space of collaboration that departs from dominant school-based approaches. These results have implications for the design of teacher evaluation policy and the furthering of equity-centered partnerships with families and communities.

Quality evaluation of digital voice assistants for the management of mental health conditions

&lt;abstract&gt;&lt;sec&gt; &lt;title&gt;Background&lt;/title&gt; &lt;p&gt;Digital voice assistants (DVAs) are gaining increasing popularity as a tool for accessing online mental health information. However, the quality of information provided by DVAs is not known. This study seeks to evaluate the quality of DVA responses to mental health-related queries in relation to six quality domains: comprehension ability, relevance, comprehensiveness, accuracy, understandability and reliability.&lt;/p&gt; &lt;/sec&gt;&lt;sec&gt; &lt;title&gt;Materials and methods&lt;/title&gt; &lt;p&gt;Four smartphone DVAs were evaluated: Apple Siri, Samsung Bixby, Google Assistant and Amazon Alexa. Sixty-six questions and answers on mental health conditions (depression, anxiety, obsessive-compulsive disorder (OCD) and bipolar disorder) were compiled from authoritative sources, clinical guidelines and public search trends. Three evaluators scored the DVAs from an in-house-developed evaluation rubric. Data were analyzed by using the Kruskal-Wallis and Wilcoxon rank sum tests.&lt;/p&gt; &lt;/sec&gt;&lt;sec&gt; &lt;title&gt;Results&lt;/title&gt; &lt;p&gt;Across all questions, Google Assistant scored the highest (78.9%), while Alexa scored the lowest (64.5%). Siri (83.9%), Bixby (87.7%) and Google Assistant (87.4%) scored the best for questions on depression, while Alexa (72.3%) scored the best for OCD questions. Bixby scored the lowest for questions on general mental health (0%) and OCD (0%) compared to all other DVAs. In terms of the quality domains, Google Assistant scored significantly higher for comprehension ability compared to Siri (100% versus 88.9%, p &amp;lt; 0.001) and Bixby (100% versus 94.5%, p &amp;lt; 0.001). Moreover, Google Assistant also scored significantly higher than Siri (100% versus 66.7%, p &amp;lt; 0.001) and Alexa (100% versus 75.0%, p &amp;lt; 0.001) in terms of relevance. In contrast, Alexa scored the worst in terms of accuracy (75.0%), reliability (58.3%) and comprehensiveness (22.2%) compared to all other DVAs.&lt;/p&gt; &lt;/sec&gt;&lt;sec&gt; &lt;title&gt;Conclusion&lt;/title&gt; &lt;p&gt;Overall, Google Assistant performed the best in terms of responding to the mental health-related queries, while Alexa performed the worst. While the comprehension abilities of the DVAs were good, the DVAs had differing performances in the other quality domains. The responses by DVAs should be supplemented with other information from authoritative sources, and users should seek the help and advice of a healthcare professional when managing their mental health conditions.&lt;/p&gt; &lt;/sec&gt;&lt;/abstract&gt;

Objective Assessment in Java Programming Language Using Rubrics

Aim/Purpose: This paper focuses on designing and implementing the rubric for objective JAVA programming assessments. An unsupervised learning approach was used to group learners based on their performance in the results obtained from the rubric, reflecting their learning ability. Background: Students' learning outcomes have been evaluated subjectively using a rubric for years. Subjective assessments are simple to construct yet inconsistent and biased to evaluate. Objective assessments are stable, reliable, and easy to conduct. However, they usually lack rubrics. Methodology: In this study, a Top-Down assessment approach is followed, i.e., a rubric focused on the learning outcome of the subject is designed, and the proficiency of learners is judged by their performance in conducting the task given. A JAVA rubric is proposed based on the learning outcomes like syntactical, logical, conceptual, and advanced JAVA skills. A JAVA objective quiz (with multiple correct options) is prepared based on the rubric criteria, comprising five questions per criterion. The examination was conducted for 209 students (100 from the MCA course and 109 from B.Tech. course). The suggested rubric was used to compute the results. K-means clustering was applied to the results to classify the students according to their learning preferences and abilities. Contribution: This work contributes to the field of rubric designing by creating an objective programming assessment and analyzing the learners’ performance using machine learning techniques. It also facilitates a reliable feedback approach offering various possibilities in student learning analytics. Findings: The designed rubric, partial scoring, and cluster analysis of the results help us to provide individual feedback and also, group the students based on their learning skills. Like on average, learners are good at remembering the syntax and concepts, mediocre in logical and critical thinking, and need more practice in code optimization and designing applications. Recommendations for Practitioners: The practical implications of this work include rubric designing for objective assessments and building an informative feedback process. Faculty can use this approach as an alternative assessment measure. They are the strong pillars of e-assessments and virtual learning platforms. Recommendation for Researchers: This research presents a novel approach to rubric-based objective assessments. Thus, it provides a fresh perspective to the researchers promising enough opportunities in the current era of digital education. Impact on Society: In order to accomplish the shared objective of reflective learning, the grading rubric and its accompanying analysis can be utilized by both instructors and students. As an instructional assessment tool, the rubric helps instructors to align their pedagogies with the students’ learning levels and assists students in updating their learning paths based on the informative topic-wise scores generated with the help of the rubric. Future Research: The designed rubric in this study can be extended to other programming languages and subjects. Further, an adaptable weighted rubric can be created to execute a flexible and reflective learning process. In addition, outcome-based learning can be achieved by measuring and analyzing student improvements after rubric evaluation.

An evaluation of educational apps for preschool-age children in Android and iOS

Applications (apps) available as educational in digital app stores for preschoolers claim that they offer to learn content on a wide range of basic academic skills, including counting, reading, and pattern recognition. This research aimed to investigate the educational quality of a range of educational applications for Android and iOS Operating Systems aimed at preschool-age children, i.e., children from 3 to 5 years old. The evaluation was done using two scientific tools: the rubrics published by Lee &amp; Cherner entitled "A Comprehensive Evaluation Rubric for Assessing Instructional Apps" and by Papadakis, Kalogiannakis &amp; Zaranis entitled "Designing and creating an educational app rubric for preschool teachers". The research results showed that most proclaimed educational applications did not have a real educational value. Research extensions are discussed.

A Thematic Analysis of the Structure of Delimitations in the Dissertation

Aim/Purpose: The purpose of the research was to examine the function and application of delimitations—what the researcher includes and excludes in a study—in the dissertation process. The aim was to map the delimitations process to improve research, rigor and relevance of findings, and doctoral completion rates using a formalized and standardized approach applied flexibly. Background: All research is bounded whether formally defined or not. Unlike limitations, which are issues which the researcher addressed after the completion of a study and cannot control, delimitations are what a researcher includes and excludes to make a project manageable and focused on the research question. Yet, there was no research identified which specifically discussed delimitations. Researching the structure and utility of delimitations in educational administration dissertations provided a systematic analysis of the formation of the scope and boundary of research in doctoral studies. Methodology: The structure of delimitations in dissertations were examined using descriptive quantitative statistics and a qualitative thematic analysis from 28 universities. The first stage included delimitations from 30 dissertations. Triangulation was conducted using the findings with a training set of delimitations in 15 dissertations with a rubric generated from the primary sample. Contribution: The thematic analysis presented a description and interpretation of the nature of delimitations and a systematic framework to improve the research process in dissertations. Mapping the delimitations process gave a detailed portrait of internal and external characteristics which could aid doctoral students in completing the dissertation. Doctoral attrition rates, poorly completed dissertations, and lack of relevance or applicability of results need remedied. Furthermore, the delimitations rubric provided a systematic method to focus communities of learners around a common goal. Findings: Findings suggested doctoral students used delimitations haphazardly and lacked a systematic application to research. Three major themes emerged from the delimitations sections: rituals, equifinality, and pragmatism. Topics within delimitations sections centered around two axes: the internal topics of sampling procedures and factors/variables and external topics of research design and conceptual/theoretical framework. Recommendations for Practitioners: Poorly understood and developed delimitations negatively impacted findings in dissertations, completion rates, and future research skills of doctoral students. By applying delimitations to a design of research framework in a community of learners, doctoral students and dissertation chairs could improve the dissertation completion process and improve research results using a Delimitations Evaluation Rubric. Recommendation for Researchers: Developing a rules-based process with a formalized and standardized process could give researchers a way to evaluate and plan the dissertation process. Developing and applying rubrics to delimitations could serve as a conduit to effective mentoring, feedback, and empowerment. Impact on Society: Improving doctoral completion rates in a timely manner would be beneficial to students’ long-term and personal interests. A well-defined delimitations process could improve the dissertation, and strengthened dissertations could add to the research base. Future Research: Delimitations are listed in one section, but the scope and boundaries are often fragmented and disjointed throughout a dissertation. By examining complete dissertations for delimitations, there could be further insight. Expanding rubrics as a tool to build a community of learners could develop a holistic approach to doctoral education.

Usefulness of rubric evaluation for practical use by nursing students

Usefulness of rubric evaluation for practical use by nursing students

Evaluating English Textbook: A Sociolinguistic Perspective

This paper examines an English textbook used in Turkey within the context of sociolinguistic issues. Various studies on book evaluation can be found in the literature; however, there seems to be very limited research analysing the course books from a sociolinguistic perspective. By employing the qualitative research design, the 3rd grade English book Just Fun English (Tiras, 2019) published for public primary schools was analysed using the Sociolinguistic Textbook Evaluation Rubric (Atar &amp;amp; Erdem, 2020). Within the framework of the six criteria of the rubric, the results indicate that the textbook is hardly in conformity with the sociolinguistic matters. Though the textbook considers the linguistic ecology of learners and makes learning English accessible to the public, there are no examples of non-native and non-standard accents of English, non-native-non-native interaction, and successful bilinguals. This study contributes to a growing body of literature by evaluating a textbook concerning the sociolinguistic issues and it offers some insights into the improvement of the textbook.

Conducting Classroom Observations: The Plight of 1:1 Laptop Title 1 Schools

ABSTRACT 1:1 laptop schools have increased over the past decade, yet minimal research exists that focuses on classroom observational practices within such schools. This study examines how teachers and school administrators from three 1:1 laptop Title 1 schools perceived standardized teacher evaluation systems and classroom observation practices. Findings revealed misaligned instructional expectations and the need to reconceptualize classroom observation supervision and evaluation tools for 1:1 laptop classrooms. Participants of this study agreed that teacher evaluation rubrics did not help identify effective instruction with technology. School administrators and policymakers must reconsider the standards and tools used to evaluate 1:1 laptop instruction.

The SPEER: An interprofessional team behavior rubric to optimize geriatric clinical care

ABSTRACT Geriatric patients with complex health care needs can benefit from interprofessional (IP) care; however, a major gap in health professional education is determining how to prepare future providers for IP collaboration. Effective IP team behavior assessment tools are needed to teach, implement, and evaluate IP practice skills. After review of IP evaluation tools, the Standardized Patient Encounter Evaluation Rubric (SPEER) was created to evaluate team dynamics in IP practice sites. Independent sample t-tests between faculty and learner SPEER scores showed learners scored themselves 15 points higher than their faculty scores (p < .001). Cronbach’s α showed high internal consistency (α = 0.91). Paired t-tests found that learners identified improvements in the team’s ability to address the patient’s education needs and to allow the patients to voice their expectations. Faculty identified improvements in the teams’ ability to make recommendations. Faculty evaluations of learner teams showed improvements in raw ratings on all but two items. Qualitative data analysis for emergent themes showed learners desired team functioning feedback and how teamwork could improve to provide optimal IP care. In conclusion, the SPEER can help faculty and learners identify growth in their teams’ ability to perform key IP skills in clinical sites.

Reflective abilities of nursing students: A thematic analysis of reflection journals

Objective: Reflective writing is consistently linked to improved clinical decision-making. However, analyzing the journals to evaluate the reflective abilities of nursing students is scanty locally. This study aimed to assess the reflective skills of undergraduate nursing students.Methods: A qualitative thematic content analysis using the Lasater Clinical Judgment Evaluation Rubric was used to assess the reflective abilities of 33 undergraduate nursing students in 138 journal entries. Guided by Gibb's reflective model, the students documented their experiences during a clinical attachment at a National Referral Hospital in Kenya between February and August 2018. Data coding and thematic linking were done using NVIVO version 11. Results: Reflective abilities differed across gender and to some extent across years of study. Most participants were more likely to notice the deviation from the norm, whether patient-related or health care environment-related. Moreover, they demonstrated the ability to respond to the situation, self-evaluate, and develop action plans for future encounters. However, the majority struggled with interpreting findings.Conclusions: Gender differences exist in the way nursing students reflect. Most nursing students focus on describing the situation rather than developing solutions. There is, however, an indication of developing reflective abilities across the year of study.

An Investigation of Anti-Black Racism LibGuides at ARL Member Institutions

This study sought to analyze anti-Black racism LibGuides created by ARL member institutions to determine strengths and weaknesses of the guides based on LibGuides best practices. Institutional and LibGuide author demographic information were also gathered to determine correlations or trends, if any. Rubric evaluation of LibGuides found that guides were strongest in areas related to guide design, materials included on the guides, and links to resources. Guides were weakest in areas related to the framing of social justice and pedagogy. Results from this study have the potential to inform the structure and revision of social justice LibGuides at a time when librarianship is grappling with issues of neutrality, racism, and becoming more anti-racist.

An Auxiliary Tasks Based Framework for Automated Medical Skill Assessment with Limited Data.

Automated medical skill assessment facilitates medical education by merging varying clinical experiences across instructors for standardizing medical training. However, medical datasets for training such automated assessment rarely have satisfactory sizes due to the cost of data collection, safety concerns and privacy restrictions. Current medical training relies on evaluation rubrics that usually include multiple auxiliary labels to support the overall evaluation from varying aspects of the procedure. In this paper, we explore machine learning algorithms to design a generalizable auxiliary task-based framework for medical skill assessment to address training automated systems with limited data. Our framework exhaustively mines valid auxiliary information in the evaluation rubric to pre-train the feature extractor before training the skill assessment classifier. Notably, a new regression-based multitask weighting method is the key to pre-train a meaningful feature representation comprehensively, ensuring the evaluation rubric is well imitated in the final model. The overall evaluation task can be fine-tuned based on the pre-trained rubric-based feature representation. Our experimental results on two medical skill datasets show that our work can significantly improve performance, achieving 85.9% and 97.4% accuracy in the intubation dataset and surgical skill dataset, respectively.