Indicators Of Student Performance Research Articles

Multidimensional Outcome Considerations in Assessing the Efficacy of Medical Educational Programs

Background: To be accredited, schools must evaluate the effectiveness of their programs. Educators are looking for specific indicators beyond the traditional measures. Data from multiple methods, including indicators of student performance and of the institutional environment, can be used to determine if educational program goals are met. Summary: This article outlines specific ways to consider three types of efficacy outcomes that are consistent with accreditation guidelines: educational, clinical career, and environmental outcomes. Specific measurable outcomes for each of these categories are derived from higher education literature: for example, learning approaches and information acquisition for education; professional behaviors and interpersonal dimensions for clinical career, and scholarship of teaching and teaching itself as a shared and valued activity for environmental outcomes. These outcomes address student assessment and program evaluation. Data from problem-based learning programs illustrate these outcomes. Conclusions: Educators can determine if educational program innovations have met their goals by collecting efficacy outcome data.

State-Mandated Curriculum Change in Three High School Physical Education Programs

High school physical education programs in South Carolina have undergone a major reform effort which was implemented through extensive staff development called the Physical Education Institute (PEI). How to teach toward student performance indicators became the focus of the in-services. This study identified factors that facilitated or hindered implementing the performance indicators, and examined the extent to which key players from three selected programs of secondary physical education were able to achieve the goals of the reform effort. Data were collected through interviews, document analysis, and videotaping lessons. Results reveal that “lead teachers” facilitated the extent to which programs met the goals of reform. The teaching/coaching role conflict was a main hindrance. Each school and teacher met each of the student performance indicators to varying degrees. Implications include the need for a shared reform, holding teachers and students accountable, and the need for administrators and university faculty to be involved.

The content and value of letters of recommendation in the resident candidate evaluative process

Purpose To determine the content and value of letters of recommendation (LOR) in the selection of residents for general surgery training. Methods During the 1999 application process, we evaluated the source, content, and usefulness of 966 LOR for 288 viable applicants to our program. Each letter was reviewed to determine if the author had clinical contact with the student and whether the student’s fund of knowledge, clinical achievements, psychomotor skills, and personal characteristics were described. The use and type of a “single word” summary was also noted. The program director then determined if the LOR was helpful in evaluating the student. Participants All LOR in the application materials of candidates applying to the General Surgery Residency of the University of Arizona. Results All of the candidates submitted 3 LOR and 34% submitted 4. The average length was 13.7 (range, 4 to 36) sentences. Department chairs wrote LOR for 66% of the students, and these comprised 20% of the total LOR. Program directors, division chiefs, and clerkship directors wrote 16%, whereas other general surgery faculty contributed 40%. In 17% of the LOR, no evidence of direct clinical supervision could be found. Personal characteristics were described in 89% of the letters, but reference to psychomotor skills occurred in only 27%. In 58% of the LOR, a single adjective was used to describe the students; of these, “outstanding” (or equivalent) was used in 37% and “excellent” (or equivalent) was used in 38%. Meaningful comparison to student colleagues appeared in 11%. In our program director’s opinion, 24% of the LOR were helpful in the evaluation process. The department chair’s letters were helpful only 19% of the time. Conclusions Letters of recommendation are imprecise indicators of student performance for resident candidate evaluation. A “standardized LOR” may help provide a consistent and more objective evaluative tool, and the APDS should consider its development for future selection processes.

The development of process indicators to estimate student gains associated with good practices in undergraduate education

The purpose of this paper is twofold. First, the development of psychometrically sound process indicators of student performance is described. Second, the results are reported of a test of the utility of these indicators with six samples of men and women students (n = 911 in each group) from 19 baccalaureate, 27 master's, and 29 doctoral-granting institutions. Process indicators represent behaviors associated with desired outcomes of college and estimate the extent to which students are engaged in these activities. Selected items from the College Student Experiences Questionnaire were used to create measures of three good educational practices: faculty-student contact, cooperation among students, and active learning. Active learning and cooperation among students in that order were the best predictors of gains for both women and men at all three types of institutions. Student background characteristics had only trivial influences on educational gains. Implications are discussed for institutional policy and further research.

Open Enrollment in Massachusetts: Why Families Choose

In 1991, the Massachusetts legislature passed an open enrollment law permitting students to enroll in schools outside their home communities. This article describes a study of enrollment patterns under the open enrollment program as of fall 1992. The study compared certain characteristics of Massachusetts sending and receiving districts in those settings in which 20 or more school-choice students transferred from one district to another. This comparison revealed that families generally enrolled their children in the schools of communities having higher indicators of student performance and higher socioeconomic status than the districts they left.

Constructing Marketing Examinations That Are Reliable and Valid Measures of Student Performance

Test construction in marketing is of ten a casual undertaking rather than a serious attempt to develop a reliable and valid indicator of student performance. This problem is exacerbated by educators’ reliance upon the test banks and computerized testing programs of ten accompanying marketing texts that typically do not address test construction issues. The authors of this paper propose a simple and systematic six-step procedure for test construction. This procedure is adapted from Churchill’s (1979) paradigm for developing better measures of marketing constructs. While the proposed procedure is initially time-consuming, its use will result in reliable, valid test instruments. Assuming there is no change in course objectives or performance evaluation measures, the resulting instrument can be utilized from one term to the next.

Reliability and efficiency of components of clinical competence assessed with five performance-based examinations using standardized patients

The present study was conducted to provide in-depth information on the reliabilities of measures of the separate components of clinical competence (e.g. data collection, test interpretation, diagnosis, etc.) assessed by a performance-based examination consisting of standardized-patient cases administered to five classes of senior medical students at Southern Illinois University School of Medicine. In general, the reliabilities of the competencies as they were actually measured on the examination (using the number of cases on which each competency was actually measured) were small, with 54% less than 0.30 and 75% less than 0.40. For generalizability coefficients pooled across the five classes and projected to a common number of k = 10 cases, two of the nine competencies had reliabilities in the 40s, a third was close to 0.40, and the remaining six were in the low 20s. The number of cases needed for the competencies to achieve the recommended reliability of 0.80 ranged from 45 to 170 cases, with six of the nine competencies requiring over 100 cases to reach the 0.80 level. The low reliabilities of these measures of the components of clinical competence raise serious questions about using the scores as indicators of student performance.

School District and School Size: An Evolving Controversy

grams, services, and personnel needed to meet modernm educational requirements? Over the last fifty years, studies that suggest optimal size have focused on cost analysis, curriculum offerings, staffing, and student achievement as the most important variables. Historically, the minimum size tends to be 10,000 to 12,000 students and the maximum size tends to be 40,000 to 50,000 students (Dawson 1934; Fantini, Gittell, and Magat 1970; Passow 1967; Summary 1974). One classic study by Mort, Vincent, and Newell (1955) considered the maximally effective school district to comprise 100,000 students. However, advocates of small school districts and rural schools argue that school districts with 5,000 student maximums are more cost effective and have fewer student dropouts (percentage wise) and higher student SAT scores, ACT scores, and high school graduation rates than school districts with more than 5,000 students (A Critique 1986). Do these national comparisons provide evidence that small schools districts are more productive? Yes, in the sense that their per pupil expenditures are not in the top 10 percent (cluster between 14 and 30 percentile), even though they rank in the top 10 percent on nationwide achievement scores and other indicators of student performance, such as fewer dropouts and more high school graduates (A Critique 1986; Sher 1981). No, in the sense that these small school districts are more homogeneous and may reflect higher socioeconomic status. The point is, however, the data ought to make us suspicious about the claim that school districts must have at least 5,000 students to achieve quality education. Historically, 10,000 students is a large number for a school district. Our schools are an outgrowth of oneroom school houses in rural America. Even as late as the

Fun and Games: Their Contribution to Basic Skills Instruction in Elementary School

Indicators of instructional program effectiveness have been expanded to include both the range of instructional options and cognitive and affective performance measures. Within the context of a statewide evaluation of educational reform, this research studied the relationship of diverse instructional materials with student performance indicators. The study sampled 72 schools from a geographically, socioeconomically, and linguistically representative frame, and two classrooms each in second and third grades of each school provided data. Observation and teacher survey data were collected, and students were tested in reading, mathematics, and attitudinal areas. Our study reports significant performance decrements associated with the use of diverse instructional resources. Explanations are provided within a framework that includes time-on-task and motivational variables.