First-time Test Takers Research Articles

Is There a Relationship Between Performance During Physical Therapist Clinical Education and Scores on the National Physical Therapy Examination (NPTE)?

Background and Purpose. The determination of whether a student physical therapist (PT) has reached entry-level status for graduation purposes is partially determined by the physical therapist education program's evaluation tool used for rating student performance in the clinical setting. The determination of whether a graduate can practice physical therapy depends on graduate performance on the National Physical Therapy Examination (NPTE). Clinical performance as assessed by the clinical evaluation tool used by members of the Texas Consortium for Physical Therapy Clinical Education (Consortium), the Physical Therapist Manual for the Assessment of Clinical Skills (PT MACS), has not been evaluated with regard to the NPTE. Therefore, the purposes of this study were to assess the following: (1) the relationship between PT MACS section scores on the first and final clinical experiences with section scores on the NPTE among first-time test takers, (2) the relationship between PT MACS aggregate scores on the first and final experiences with the overall score on the NPTE among first-time test takers, and (3) the relationship of final PT grade point averages (GPAs) with NPTE scores among first time test takers. Subjects/Methods. This retrospective study used archived academic, clinical, and NPTE performance data collected by the directors of Clinical Education of 8 Texas PT professional education programs for classes graduating between 2004 and 2007. The total number of graduates was 985. Data were analyzed for relationships using Pearson product moment correlation coefficients. Results. Small, but significant, relationships were found between 2 PT MACS sections and corresponding sections on the NPTE (P Discussion and Conclusion. Although-several small, significant relationships were found between the PT MACS sections and corresponding NPTE sections, PT GPA demonstrated the greatest relationship with success on the NPTE, reinforcing the findings of previous studies. Based on results of the inter-item analysis, the PT MACS demonstrated good internal consistency reliability. Results of this study suggest that an assessment tool that tests all domains of learning cannot predict performance on a cognitive instrument such as the NPTE. Key Words: Physical therapy, Clinical education, Licensure examination. BACKGROUND AND PURPOSE Today's health care system requires that newly graduated physical therapists (PTs) enter the workforce prepared to meet the demands of total patient management. Graduates are expected to perform at a level that meets the standards of safe, effective, and efficient practice. The educational preparation of these graduates includes completion of both intense schedules of didactic classroom experience and complex clinical experiences. The combination of didactic preparation and clinical competence in entry-level skills must be assessed in order to determine the eligibility of a student to graduate from the PT education program. Clinical competence is assessed using evaluative tools specific to clinical skills that define entry-level standards. One such evaluative tool is the Physical Therapist Manual for the Assessment of Clinical Skills (PT MACS),1 developed by the Texas Consortium for Physical Therapy Clinical Education. Successful completion of an accredited professional physical therapist education program allows the graduate to apply for and take the National Physical Therapist Examination (NPTE). Entry-level competence is assessed using multiple-choice items that have been constructed based on a formal survey of physical therapists describing their current job activities and responsibilities. …

Investigating Repeater Effects on Chained Equipercentile Equating With Common Anchor Items

This study investigated the impact of repeat takers of a licensure test on the equating functions in the context of a nonequivalent groups with anchor test (NEAT) design. Examinees who had taken a new, to-be-equated form of the test were divided into three subgroups according to their previous testing experience: (a) repeaters who previously took the reference form, to which the new form would be equated; (b) repeaters who previously took any form other than the reference form; and (c) first-time test-takers for whom the new form was the first exposure to the test. Equating functions remained essentially invariant across all repeaters versus first-time test-takers, supporting score equatability of the two forms. However, when the repeater subgroup was sub-divided based on the particular form examinees took previously, subgroup equating functions substantially differed from the total-group equating function, indicating subgroup dependency of score equating. The results indicate that repeater membership needs to be more clearly specified to assess the impact of repeaters on score equating. Such clarification may be especially necessary for high-stakes licensure tests because repeaters tend to perform more poorly on such tests than first-time test-takers.

Teacher Licensure Exams and Black Teacher Candidates: Toward New Theory and Promising Practice

Basic teacher licensure exams such as Praxis are the first gatekeepers to the teaching profession. Fewer than half of the aspiring Black teachers who take these exams pass on their first attempt. While critiques of these exams are warranted, critiques alone will do little to help certify more Black teachers. This solution-oriented article makes both a theoretical and practical contribution to this area. First, it provides a selected overview of two theoretical areas that have relevance to understanding teacher licensure testing. These theoretical areas are self-efficacy and sociocultural theory. The article then draws from the author 's 6 years of preparing Black preservice teachers for licensure exams to illustrate how these theories can shape practice at both programmatic and classroom levels.Keywords: teacher licensure, teacher education, self-efficacy, sociocultural theory, Praxis, Black and African American teachersAny discussion of increasing the numbers of Black teachers in United States' schools must include a discussion of when and how prospective teachers are tested. At present time, standardized teacher licensure examinations, which are designed and administered by testing agencies such as Educational Testing Service (ETS; www.ets.org/praxis) and Pearson (www.pearsonassessments.com) are gatekeepers to most teacher education programs in the United States. These exams entail a battery of timed tests in reading, mathematics, and writing. Currently, 29 states use ETS' Pre-Professional Skills Test (i.e., Praxis I) as a basic skills admissions test for teacher education programs. Twelve other states use a similar test created by a different testing company. Summarily, in 41 states, students must pass a licensure examination before they are admitted into a program. Some universities whose state department of education does not have these requirements still elect to have these exams as program admissions requirements.Due to the position of these exams, education is an undergraduate major fundamentally different from others because students must test into it. While it is normal for state certifying bodies in many professions (e.g., law, medicine) to administer professional assessments before granting licensure, the organization of teacher licensure is different from other professions. Traditionally, teacher education and licensure take place at the undergraduate level and not the post-baccalaureate level as in law and medicine. Consequently, prospective teachers must pass basic skills licensure exams before they can be formally admitted into a program, usually during their first or second year at an institution.Research by ETS (Nettles et al, 201 1) found that the first time passing rates of Black test takers between 2005 and 2009 on the Praxis basic skills exam were as follows: reading (40.7%), writing (44.2%), and mathematics (36.8%). Nettles and colleagues (2011) found that the passing rate for White first-time test takers was approximately double that for Black test takers: reading (81.5%), writing (79.5%), and mathematics (78.2%). Even when controlling for other variables such as household income, grade point average, and parent's education level, a 7-point math score difference still exists between Black and White test takers. While the passing rates and standards for White students should not be held as the desired standard for Black students, this gap is alarming and is a critical piece to understanding why only 7% of teachers in the United States are Black (National Center for Education Information, 201 1). If one cannot be admitted into a teacher education program, one cannot become a teacher. This battery of tests is the first significant gatekeeper that students who wish to become teachers experience in postsecondary education.Critiques abound for such standardized tests in general and their contemporary placement as gatekeepers in teacher education programs (Bennett, McWhorter, & Kuykendall, 2006; Flippo, 2003; Memory, Coleman, & Watkins, 2003; Mikitovics & Crehan, 2002). …

United States Medical Licensing Examination Step 1 Two-Digit Score: A Correlation With the American Board of Pathology First-Time Test Taker Pass/Fail Rate at the University of Pittsburgh Medical Center

Factors that correlate with success or failure on the American Board of Pathology (ABP) examination are not known. Other medical residency programs have shown that standardized test scores correlate with specialty board examination scores; however, data from pathology programs are lacking. To investigate whether the 2-digit score on step 1 of the United States Medical Licensing Examination (USMLE) was correlated with ABP examination performance at a large university pathology program. Nine years of data (2001-2009) from pathology residents (n = 72) at the University of Pittsburgh Medical Center (UPMC, Pittsburgh, Pennsylvania) was collected from existing files and deidentified. Step 1 USMLE 2-digit scores and ABP failure rates for first-time test takers were compared. Results are reported as the percentage of residents who failed either the anatomic pathology or clinical pathology part of the ABP examination in cohorts by their USMLE 2-digit score (≤80, 81-85, 86-89, ≥90). The rolling 5-year (2005-2009) ABP average failure rate for first-time test takers of the anatomic pathology examination was 3.1% (UPMC) and 14.1% (nationally); in clinical pathology, it was 13.8% (UPMC) and 23.6% (nationally). At UPMC, no resident failed the anatomic pathology or clinical pathology parts of the ABP examination if his or her 2-digit USMLE step 1 score was 90 or more across 9 years of training (2001-2009). In the UPMC pathology program, 2-digit scores on USMLE step 1 of 90 or more and 80 or less were strong measures of ABP first-time pass/failure rates, whereas scores of 81 to 89 were less-accurate measures. The USMLE step 1 score is one of many criteria that can be used for screening applicants for a pathology residency program.

Evolution of Student Assessment Following Implementation of an Integrated Medical Curriculum: Contribution to Improved Educational Outcomes

This paper describes the stimuli and rationale for and the nature of assessment changes implemented after embarking on a new student-centered, problem-based learning curriculum. In 1998, the University of Texas Medical Branch implemented the Integrated Medical Curriculum (IMC), a problem-based curriculum with sequential, interdepartmental, clinically relevant basic science courses and a concurrent Practice of Medicine course. The IMC’s aim was to improve educational quality by emphasizing knowledge application over rote memorization, student-directed learning, problem-solving skills development, basic science and clinical concepts integration, early acquisition of clinical skills, and professional teamwork. During the initial years of the IMC, students’ learning outcomes did not improve over those of the former curriculum because, in part, we did not initially modify student assessment (a driving force for student study habits) to align with the new curricular goals and philosophies. We subsequently took several steps to modify our assessment practices in order to improve student learning and understanding. These steps included (1) increasing the number and quality of higher -order exam questions, (2) sequestering examinations, (3) increasing the number of practice exam items in each course, (4) and emphasizing in-depth discussion in problem-based learning sessions by introducing small-group quiz exercises. Following the assessment modifications, our students’ USMLE Step 1 scores and our pass rate for first-time test takers rose from being consistently below the national Step 1 average to above the national average. This improvement has been sustained for the past seven years. We believe that implementing those assessment modifications within the framework of the newly devised student-centered, problem-based curriculum in an interdepartmental environment resulted in a critical mass of “local” changes. The changes, initiated and supported by students, faculty and administration, catalyzed, produced and sustained more “global” changes that improved students’ learning, thus better preparing them for the USMLE Step 1 examination.

Test-Specificity of the Advantage of Retaking Cognitive Ability Tests

In selection settings, when people retake the same cognitive ability tests, scores are generally positively biased. Our approach aimed to test whether these previous exposure effects are test-specific or transferable to other tests. We compared the differences between scores for first-time test takers and retakers for 2 kinds of material: ‘old’ tests, known only to the retakers, and ‘new’ tests, unknown to both groups. The current study used data collected during 2 sessions –S and S+1 – of a selection process for entry into the French national air transport pilot training system, with at least 500 first-time test takers and 130 retakers in each session. For Session S, on average, retakers scored higher on the ‘old’ tests, but not on the ‘new’ tests. Moreover, the material that was new to retakers at Session S was old at Session S+1, and the finding for old tests could be replicated at Session S+1. The finding that the acquired skills that led to higher scores on old tests were only test-specific is discussed.

Emergency Medicine Oral Board Review Illustrated

Emergency Medicine Oral Board Review Illustrated

Low Correlation between Subjective and Objective Measures of Knowledge on Surgery Clerkships

Medical student knowledge is assessed during surgical clerkships subjectively and objectively. Subjective evaluation depends on faculty assessment during clinical and didactic interactions. Objective measurement derives from standardized tools, such as the National Board of Medical Examiners Surgery Subject test (shelf). Few efforts have been made to characterize the correlation between subjective and objective measures of medical knowledge. All 308 third-year medical students who completed the 8-week surgery clerkship at the University of North Carolina at Chapel Hill between July 2005 and June 2007 received subjective assessment of knowledge on 3 clinical rotations (one 4-week core and two 2-week elective rotations) and a longitudinal small-group tutorial. Faculty evaluators assigned percentile scores to rate students' knowledge base relative to their peers. In addition, students took the shelf test the last day of clerkship, and percentile scores were assigned based on National Board of Medical Examiners-supplied normative data from first-time test-takers within the same academic quarter. Subjective versus objective knowledge scores were plotted overall, and Pearson product-moment correlation coefficients were generated for core, elective, and tutorial assessments. There were only weak linear relationships noted between subjective faculty-assigned knowledge scores and objective shelf scores. Pearson correlations were 0.24 for core rotations (4 weeks exposure), 0.14 for elective rotations (2 weeks exposure), and 0.22 for tutorials (1-hour exposure/week during 8 weeks), with p values <0.0001. Faculty assessment of knowledge is only weakly correlated with shelf performance. Faculty evaluations after 4-week rotations or longitudinal small-group interactions are better correlated with shelf scores than after 2-week electives.

National Asthma Educator Certification: Purpose, Progress, and Promise

RATIONALE: In this abstract we describe the need, development, and evaluation of a national examination to document excellence in asthma education through the Certified Asthma Educator process.METHODS: The National Asthma Educator Certification Board (NAECB) was incorporated in February 2000 following a Stakeholder Consensus Conference held in Washington, D.C. in January 1999 to develop a voluntary asthma educator certification examination consistent with the needs of a diverse set of asthma educator service providers; and to document excellence in asthma education and self-management based on the NAEPP Expert Panel Reports (EPR) published in 1991, 1997, and 2007. Comprehensive Asthma Educator Job Tasks Surveys were completed in 2001 and again in 2008 to develop and to update a comprehensive content outline for the construction of relevant, reliable, and valid test items based on current, scientifically sound concepts of disease management (i.e., asthma pathophysiology, patient and family assessment, asthma management, and program outcomes).RESULTS: There are currently 2,792 certified asthma educators; most test-takers (75%) include nursing professionals and respiratory therapists. Lifetime pass rates for first-time test takers is 67.8%, a value lower than that for pass rates for the CRT examination (75% in 2008), the CHES examination (79.2% in 2008), and the ANP examination (90.2% in 2007). Nationally, reimbursement for asthma education is limited in some cases to those who demonstrate the Certified Asthma Educator (AE-C) credential.CONCLUSIONS: AE-Cs are recognized for their knowledge of asthma management and control through successful completion of a rigorous, evidence-based examination that matches health care workforce tasks and expert recommendations. RATIONALE: In this abstract we describe the need, development, and evaluation of a national examination to document excellence in asthma education through the Certified Asthma Educator process. METHODS: The National Asthma Educator Certification Board (NAECB) was incorporated in February 2000 following a Stakeholder Consensus Conference held in Washington, D.C. in January 1999 to develop a voluntary asthma educator certification examination consistent with the needs of a diverse set of asthma educator service providers; and to document excellence in asthma education and self-management based on the NAEPP Expert Panel Reports (EPR) published in 1991, 1997, and 2007. Comprehensive Asthma Educator Job Tasks Surveys were completed in 2001 and again in 2008 to develop and to update a comprehensive content outline for the construction of relevant, reliable, and valid test items based on current, scientifically sound concepts of disease management (i.e., asthma pathophysiology, patient and family assessment, asthma management, and program outcomes). RESULTS: There are currently 2,792 certified asthma educators; most test-takers (75%) include nursing professionals and respiratory therapists. Lifetime pass rates for first-time test takers is 67.8%, a value lower than that for pass rates for the CRT examination (75% in 2008), the CHES examination (79.2% in 2008), and the ANP examination (90.2% in 2007). Nationally, reimbursement for asthma education is limited in some cases to those who demonstrate the Certified Asthma Educator (AE-C) credential. CONCLUSIONS: AE-Cs are recognized for their knowledge of asthma management and control through successful completion of a rigorous, evidence-based examination that matches health care workforce tasks and expert recommendations.

Performance Improvement Assessment of the Interservice Physician Assistant Program (IPAP)

Purpose. The Interservice Physician Assistant Program (IPAP) located at Fort Sam Houston, Texas, provides the education for the majority of all Army, Navy, and Air Force physician assistants. As such, ensuring that program maintains premier performance to support combat and peacetime healthcare is vital to the interests of the Department of Defense. To support this mission, an analysis of structure, process, and outcomes for the program was conducted using survey data and Physician Assistant National Certifying Examination (PANCE) scores. Methods. Methods employed included Chi Square analysis, regression, small multiple comparisons. Graphical and statistical results provided evidence of program strengths and weaknesses.Results. Analysis of survey data (33% response rate) revealed that students who had graduated four or more years ago were more likely to self-report that IPAP prepared them for clinical work (Chi-Square = 8.16, p<.006, Odds Ratio: 3.1:1). Students reported high-confidence in the disciplines of EENT, orthopedics, and dermatology as well as low-confidence in hematology and dentistry. Analysis of performance deltas provided statistically significant and practically relevant results (F(16,48) = 17.59, p < .001, adj. R^2 = .806). IPAP student first-time test-takers routinely performed above the national average; however, performance in recent years has declined. Areas of strength identified in the analysis included EENT and dermatology confirming student self-assessments. One area of weakness identified was psychiatry and behavioral health.Conclusions. Based on the analysis of structure, process, and outcome, IPAP leadership is evaluating specific performance improvement initiatives to maintain program strengths and address weaknesses.

The Effectiveness of Streaming Video on Medical Student Learning: A Case Study

Information technology helps meet today’s medical students’ needs by providing multiple curriculum delivery methods. Video streaming is an e-learning technology that uses the Internet to deliver curriculum while giving the student control of the content’s delivery. There have been few studies conducted on the effectiveness of streaming video in medical schools. A 5-year retrospective study was conducted using three groups of students (n = 1736) to determine if the availability of streaming video in Years 1-2 of the basic science curriculum affected overall Step 1 scores for first-time test-takers. The results demonstrated a positive effect on program outcomes as streaming video became more readily available to students. Based on these findings, streaming video technology seems to be a viable tool to complement in-class delivery methods, to accommodate the needs of medical students, and to provide options for meeting the challenges of delivering the undergraduate medical curriculum. Further studies need to be conducted to continue validating the effectiveness of streaming video technology.

The effectiveness of streaming video on medical student learning: a case study.

Information technology helps meet today's medical students’ needs by providing multiple curriculum delivery methods. Video streaming is an e-learning technology that uses the Internet to deliver curriculum while giving the student control of the content's delivery. There have been few studies conducted on the effectiveness of streaming video in medical schools. A 5-year retrospective study was conducted using three groups of students (n = 1736) to determine if the availability of streaming video in Years 1–2 of the basic science curriculum affected overall Step 1 scores for first-time test-takers. The results demonstrated a positive effect on program outcomes as streaming video became more readily available to students. Based on these findings, streaming video technology seems to be a viable tool to complement in-class delivery methods, to accommodate the needs of medical students, and to provide options for meeting the challenges of delivering the undergraduate medical curriculum. Further studies need to be conducted to continue validating the effectiveness of streaming video technology.

KATTS: A Framework for Maximizing NCLEX-RN Performance

A key indicator of the quality of a nursing education program is the performance of its graduates as first-time takers of the NCLEX-RN. As a result, nursing schools are open to strategies that strengthen the performance of their graduates on the examination. The Knowledge base, Anxiety control, Test-Taking Skills (KATTS) framework focuses on the three components of achieving a maximum score on an examination. In KATTS, all three components must be present and in proper balance to maximize a test taker's score. By strengthening not just one but all of these components, graduates can improve their overall test scores significantly. Suggested strategies for strengthening each component of KATTS are provided. This framework has been used successfully in designing remedial tutoring programs and in assisting first-time NCLEX test takers in preparing for the licensing examination.

Trauma Education and Care in the Jungle of Ecuador, Where There is no Advanced Trauma Life Support

The advanced trauma life support course is not available or affordable to rural areas in low-income countries. A trauma continuing education course was created to educate physicians of rural hospitals in the jungles of Ecuador. A basic trauma care course was designed based on local resources and location of injury, including rudimentary health posts in the jungle, rural hospitals, and definitive referral centers. Course effectiveness was evaluated by a comparison of test scores before and after the course. A multiple choice questionnaire was given. Comparison to previous test scores was also performed. Paired Student's t test was used for statistical analysis. An objective structured clinical examination (OSCE), based on the course design, was administered. Twenty-six rural physicians participated in the course. Mean test scores significantly improved from pretest to post-test (72% to 79%; p = 0.032). Knowledge deficiencies in prehospital care, extremity injury care, and patient evaluation adjuncts significantly improved from 23% to 87%, 23% to 100%, and 31% to 100%, respectively. Test results after the course showed improvements in all major categories tested. Twelve of the 26 participants were repeat test takers from a course provided 2 years earlier. These participants showed improved pretest scores compared with their highest previous test score (76.8% versus 68.5%; p = 0.0496). Compared with first-time test takers, these participants showed improved pretest (76.8% versus 68.4%) as well as post-test (81% versus 76%) scores. Twenty-five of the 26 physicians participated in the OSCE, with a pass rate of 76%. The OSCE identified various strengths and deficiencies based on patient location and available resources. In rudimentary health posts, management was adequate for hemorrhage control (65%), immobilization (77%), and early transfer to rural hospitals (92%). Prehospital communication was inadequate (53%). Rural hospital management was adequate for primary evaluation (60%) and resuscitation (74%) but poor in secondary patient evaluation (53%), adjuncts (25%), and transfer to definitive referral centers (11%). OSCE scores differed from multiple choice questionnaire test results. Where there is no advanced trauma life support, a tailored trauma course and evaluation can be effective in educating local providers. A well-designed competency evaluation (multiple choice questionnaire and OSCE) is helpful in identifying deficient local aspects of trauma care. The course design and evaluation methods may serve as a model for continuing trauma care education in developing countries.

Ensuring NCLEX-RN Success for First-Time Test-Takers

A major indicator of baccalaureate nursing program effectiveness is the pass rate on the National Council Licensure Examinations, Registered Nurse (NCLEX-RN) among first-time candidates. This article highlights the scope, instructional methods, and outcomes of the remediation program at Georgetown University School of Nursing & Health Studies (GU-NHS). With implementation of the multifaceted program, the NCLEX-RN pass rate for first-time takers at GU-NHS has been consistently above the national mean among all first-time candidates. Although it is impossible to determine which strategy made the greatest contribution to the success of the program, the cumulative effect is impressive and indicates the students are prepared for the challenge of the examination and contemporary nursing practice.

Passing the Boards

Klein, Gregg R. MD1; Austin, Matthew S. MD1; Randolph, Susan1; Sharkey, Peter F. MD1; Hilibrand, Alan S. MD1 Author Information