Development Of Rubric Research Articles

Engaging prospective English teachers in the rubric development and the evaluation of a multimedia-assisted language learning textbook

This study examines pre-service English teachers'participation in the development of a rubric for the evaluation of a multimedia-assisted language learning textbook and explores learner-directed evaluation of the textbook constructed by learners The current research aims to investigate: 1) the learners'response as textbook evaluators, 2) the number of analytic areas the learners describe in their own evaluation rubric and the area that the learners give weight on in evaluating the software and 3) the inter-rater differences across the areas and across the four types of evaluation-(self and peer assessment, peer assessment using the self-developed rubric, self assessrnent using the peer developed rubric. Twenty- eight students participated in the study. The results suggest that leamers point out'instmctional design/maintenance'as the most valued area and that inter-rater difference is small in the area of 'goal/objectives' and in the context of self assessment. Discussions and implications are presented.

Developing and Using Rubrics in Occupational Therapy

Rubrics, the topic of the first Collaboration and Connections column, are an authentic assessment method used by teachers to measure student performance on classroom tasks. As an occupational therapy intervention strategy, rubrics are easy to integrate within the classroom environment. Occupational therapists can use rubrics to support classroom performance, define performance expectations, and measure progress on goals. Rubrics can also be a self-evaluation tool for students. Rubric development involves collaborating with the teacher and student to: define the purpose of the rubric, analyze current performance, identify performance components, select a measurement scale, and define performance criteria.

VALUE: Valid assessment of learning in undergraduate education

This chapter provides an example of a tool developed by the American Association of Colleges and Universities that colleges and universities can use to assess student learning through the development of rubrics.

Extending the Responsibilities for Schools beyond the School Door

In this article, the authors use qualitative and quantitative research to identify the performance of social actors whose decisions impact students and teachers, elucidate a set of measurements for their performance, and offer both a theoretical and research justification for these measurements. The work challenges two faulty assumptions behind the No Child Left Behind (NCLB) legislation that make it more likely that school curricula will continue to be unrepresentative of diverse experiences and that far too many children will continue to attend schools under unnecessarily trying conditions. The first faulty assumption is the legislation's location of school ‘problems’ or the ‘problems with schools’ as beginning and ending at the school door. A second assumption that the authors’ development of research-based rubrics seeks to challenge is a prevalent attitude in US society regarding individual responsibility for personal success or failure, which supports the thrust of NCLB in the public imagination. Like the ill-distribution of economic possibilities despite people's hard work, rubrics holding the various public stakeholders in education are required to appropriately expand responsibility for student success beyond schools and teachers.

Clinical Judgment Development: Using Simulation to Create an Assessment Rubric

Clinical judgment is a skill every nurse needs, but nurse educators sometimes struggle with how to present it to students and assess it. This article describes an exploratory study that originated and pilot tested a rubric in the simulation laboratory to describe the development of clinical judgment, based on Tanner's Clinical Judgment Model.

Using Rubrics for Assessing Student Projects in FAR Part 147 Programs

Most educators in Aviation Maintenance Technology (AMT) programs will agree that the assessment and grading of practical projects in the lab portion of their AMT courses should be as objective as possible. The U.S. Department of Transportation (USDOT) and the Federal Aviation Administration (FAA) have published Advisory Circular (AC) 147-3A, to provide guidance to aviation maintenance technician schools operating under Part 147 of Title 14 of the Code of Federal Regulations (14 CFR), commonly known as the Federal Aviation Regulations (FAR). According to AC 147-3A, in the development of practical projects, objective grading criteria must be used. The challenge for the instructor is to develop practical solutions for the objective grading of projects, and the purpose of this paper is to suggest one such practical solution – the development and use of rubrics.

Standards-Referenced Assessment for Vocational Education and Training in Schools

This study examined a model of assessment that could be applied nationally for Year Twelve Vocational Education and Training (VET) subjects and which could yield both a differentiating score and recognition of competence. More than fifty colleges across all states and territories of Australia field-tested the approach over one school year. Results showed that the model allowed for a standards-referenced model to be used: that the approach was compatible with the diverse range of senior secondary assessment systems in use throughout Australia and that there were considerable cost benefits to be had in adopting the logic of item response modelling for the development of rubrics for scoring performances on units of competence from National Training Packages. A change in the logic of competency assessment was proposed, in that the performance indicators were not rated using a dichotomy but with a series of quality ordered criteria to indicate how well students performed specified tasks in the workplace or its simulation. The study validated the method of assessment development, demonstrated the method's consistency, and showed how the method could address the issue of consistency across states. The study also proposed a set of principles for a joint assessment of both quality and competence.

What Do Students Gain by Engaging in Socioscientific Inquiry?

The question of what students gain by engaging in socioscientific inquiry is addressed in two ways. First, relevant literature is surveyed to build the case that socioscientific issues (SSI) can serve as useful contexts for teaching and learning science content. Studies are reviewed which document student gains in discipline specific content knowledge as well as understandings of the nature of science. SSI are also positioned as vehicles for addressing citizenship education within science classrooms. Although the promotion of citizenship goals seems widely advocated, the specifics of how this may be accomplished remain underdeveloped. To address this issue, we introduce socioscientific reasoning as a construct which captures a suite of practices fundamental to the negotiation of SSI. In the second phase of the project, interviews with 24 middle school students from classes engaged in socioscientific inquiry serve as the basis for the development of an emergent rubric for socioscientific reasoning. Variation in practices demonstrated by this sample are explored and implications drawn for advancing socioscientific reasoning as an educationally meaningful and assessable construct.

A Reconception of Performance Study in the Philosophy of Music Education

The actual placeof performance in music education has been the subject of nu-merous debates over the years. Most debates have revolved within the paradigmof the performance ability of the teacher and consequently the performance abil-ity of the students. Is the level to be attained that of a winning concertband/marching band/choir? Or, is the level to be attained the level at which thestudents and teacher feel the most comfortable and pleased? Most of this debate,along with the development of various assessments (including the Watkins-Farnum Test) and various rubrics to score music performance competitions, hasemphasized characteristics of performance pertaining to accuracy of notes, pre-cision of rhythm, and attendance to musicianship, style, and interpretation. Onthe other hand, the aesthetic value of performance has also been questioned inthat the value of music in the schools should not be assessed solely by the impor-tance placed on the attainment of performance skills by the students, suggestingthat it is only through performance that one may know music. For example, in arecent article in the Music Educator’s Journal,

Rubrics: Tools for Making Learning Goals and Evaluation Criteria Explicit for Both Teachers and Learners

Introduction of new teaching strategies often expands the expectations for student learning, creating a parallel need to redefine how we collect the evidence that assures both us and our students that these expectations are in fact being met. The default assessment strategy of the typical large, introductory, college-level science course, the multiple- choice (fixed response) exam, when used to best advantage can provide feedback about what students know and recall about key concepts. Leaving aside the difficulty inherent in designing a multiple-choice exam that captures deeper understandings of course material, its limitations become particularly notable when learning objectives include what students are able to do as well as know as the result of time spent in a course. If we want students to build their skill at conducting guided laboratory investigations, developing reasoned arguments, or communicating their ideas, other means of assessment such as papers, demonstrations (the “practical exam”), other demonstrations of problem solving, model building, debates, or oral presentations, to name a few, must be enlisted to serve as benchmarks of progress and/or in the assignment of grades. What happens, however, when students are novices at responding to these performance prompts when they are used in the context of science learning, and faculty are novices at communicating to students what their expectations for a high-level performance are? The more familiar terrain of the multiple-choice exam can lull both students and instructors into a false sense of security about the clarity and objectivity of the evaluation criteria (Wiggins, 1989 ) and make these other types of assessment strategies seem subjective and unreliable (and sometimes downright unfair) by comparison. In a worst-case scenario, the use of alternatives to the conventional exam to assess student learning can lead students to feel that there is an implicit or hidden curriculum—the private curriculum that seems to exist only in the mind's eye of a course instructor. Use of rubrics provides one way to address these issues. Rubrics not only can be designed to formulate standards for levels of accomplishment and used to guide and improve performance but also they can be used to make these standards clear and explicit to students. Although the use of rubrics has become common practice in the K–12 setting (Luft, 1999 ), the good news for those instructors who find the idea attractive is that more and more examples of the use of rubrics are being noted at the college and university level, with a variety of applications (Ebert-May, undated; Ebert-May et al., 1997 ; Wright and Boggs, 2002 ; Moni et al., 2005 ; Porter, 2005 ; Lynd-Balta, 2006 ).

‘Them and us’: Teachers' and tutors' experiences of a ‘new’ professional development course in Greece

The research reported in this paper compares the experiences of twenty‐two secondary teachers and twelve university tutors as they embark on a course of a ‘new’ in‐service professional development programme in Greece, in relation to adult learning. It harnesses a qualitative methodology and draws upon a set of ideas that cohere under the rubric programme development to build up a heuristic guiding research technique and analysis procedure. The analysis focuses on two key parameters of programme development for adults i.e. climate and planning, and on their associated elements. The findings identify the ‘them’ and ‘us’ image that continues to perpetuate the ‘new’ courses and indicate some tentative points that might have implications for the design of the in‐service training of teachers.

Standardizing the interview process and developing a faculty interview rubric: An effective method to recruit and retain online instructors

With the increasing prevalence of online education, institutions of higher education may encounter challenges as they seek to hire instructors who have the skills, knowledge base, and competencies that “best fit” the unique characteristics associated with online teaching. This article describes the initial development of the Systematic Interview Rubric for Adjunct Applicant Review (SIRAAR) as part of a standardized interview process for hiring online faculty members at an online university. The development of the SIRAAR stemmed from the findings of a qualitative study using a web board focus group with 28 online faculty from Capella University, School of Human Services. Once an initial iteration of the SIRAAR was completed, it was then reviewed utilizing a “vetting” process with 11 faculty members in an informal, face-to-face discussion. The use of an interview rubric and a standardized interview process is proposed as a systematic strategy to support the hiring process, online faculty retention and quality online instruction.

Instructional Module in Fourier Spectral Analysis, Based on Principles of “How People Learn”

AbstractThis paper describes the design and evaluation of an instructional module for teaching/learning Fourier spectral analysis, with emphasis on biomedical applications. The module is based on the principles of “How People Learn” (HPL) as embodied in the Legacy cycle. This cycle is a particular instantiation of problem‐based learning and includes components explicitly aimed at providing context and motivation, facilitating exploration, developing in‐depth understanding, and incorporating opportunities for self‐assessment. In the spectral analysis module, traditional teaching methods are augmented with small group discussions, peer‐to‐peer learning, a Web‐based tutorial, and an interactive demonstration. Assessment included the development of rubrics for scoring student understanding of key concepts, revealing that students who used the module demonstrated better understanding relative to students who studied the material using traditional methods. Survey results and comments indicate that students generally liked the interactive tutorial and demonstration, as well as the structure provided by the HPL framework.

Assessing Students' Problem‐Solving Assignments

AbstractProfessors can use rubrics as a valuable evaluation tool with probelm‐based learning activities. In this article, we guide professors through the process of rubric development.

A Comparison of Teachers' Assessment Practices in School Restructuring Models by Year of Implementation

Teachers' assessment practices were investigated in the context of school restructuring in Title I schools. The survey method included questionnaires distributed to teachers in 11 elementary schools in their 1st year of implementation and teachers in 11 elementary schools in their 4th year of implementation. Focus group interviews were conducted with groups of 8-10 teachers at each school. Results indicated that schools in their 4th year of restructuring had significantly higher mean ratings on the alternative assessment items than did schools in their 1st year of restructuring. These differences were significant for portfolios and student self-assessments. There were significant, positive correlations between scores on the alternative assessment scale with scores on the pedagogical change and student outcome scales. The qualitative data also suggested an increase in teachers' use of alternative assessment strategies and the development of rubrics to evaluate these assessments. There were more responses indicating changes in assessment among teachers in their 4th year of restructuring than among teachers in the 1st year. The qualitative data further indicated that teachers were concerned with the incompatibility between the alternative, authentic models advocated in the restructuring models and the district and state accountability systems that relied on standardized objective tests.

Alternatively Assessing an In‐service Program

In‐service evaluation is typically inadequate. Specifically, most in‐service assessments provide a limited amount of information about the in‐service program, and the participant does not take an active role in the evaluation. One possible first step to reforming in‐service evaluation is integrating the assessment process with the in‐service program. This paper describes how the staff in one in‐service program attempted to integrate assessment with an in‐service program through the development and utilization of an implementation rubric. At the conclusion of the in‐service program, the developed rubric provided program staff with information about the effectiveness of the in‐service program. Yet throughout the in‐service program, the rubric provided participants with opportunities to reflect upon their practice, revealed the changes that occurred in participants' classrooms and their instruction, and guided the instructional decisions of program staff.

STANDARDS‐BASED SCORE INTERPRETATION: ESTABLISHING VALID GROUNDS FOR VALID INFERENCES

ABSTRACTThe construct validity of content standards is addressed in terms of their representative coverage of a construct domain and their alignment with the students' cognitive level of developing expertise in the subject matter. The construct validity of performance standards is addressed in terms of the extent to which they reflect increasing levels of construct complexity as opposed to construct‐irrelevant difficulty. Also critical is the extent to which performance standards characterize the knowledge and skills operative at each level both to accredit specific accomplishment and to serve as goals for further learning. All of this depends on construct‐valid assessment attuned to the content standards and the development of dependable scoring rubrics and measurement scales for representing the performance standards.

Learning-based assessments of history understanding

Research undertaken to develop measures of deep understanding of American history for secondary school students serves as a method to study performance assessment in general. The assessment-policy context conditions conceptual and technical choices in design, reinforcing the importance of linking large-scale performance assessment and classroom uses. Using design perspective constraints such as desired instructional change, the state of agreement in the field of history and the practicality of assessment for wide-scale use are considered. A model of understanding adapted from Kintsch's (1992) construction-integration theory underlies the assessment. Performance differences between expert and novice explanations of primary historical sources were used as the basis of the development of a general scoring rubric. The task syntax—including a measure of prior knowledge, literal comprehension of primary texts, and an extended written explanation—has been applied to six different history topics with comparable results. Technical research is briefly summarized, including the development and validation of the scoring rubric, and its application to different history topics.