College Admission Test Scores Research Articles

Development and evaluation of additional college of pharmacy admittance variables: Life experiences and holistic composite scores

IntroductionThis study aimed to develop and evaluate new variables for assessment of doctor of pharmacy (PharmD) program candidates. MethodsTwo new admissions variables were created. The life experiences variable included number of awards, letter of reference scores, degree attainment, publications, prior academic suspension, leadership experience, work experience, and a diversity score. The holistic composite variable consisted of Pharmacy College Admission Test score, pre-pharmacy grade point average (GPA), interview score, and the new life experiences score. Variables were retroactively applied to prior admissions data, and scores of successful applicants were compared to scores of unsuccessful applicants. ResultsA total of 968 applicants from the 2015 and 2016 applicant pools were included in the evaluation analysis. Applicants offered admission had higher average scores for life experiences and holistic composite variables compared to those that were not offered admission (63.6 vs. 59.95 and 77.3 vs. 64, respectively). The life experiences and holistic composite scores were positively correlated with an offer of admission (0.142 and 0.547, respectively) and post-admission GPA (0.211 and 0.433, respectively) and negatively correlated with applicants accepting the offer of admission (−0.158 and − 0.25, respectively). ConclusionsThe holistic composite score was strongly predictive of an offer for admission and post-admission academic success, indicating that it may be a useful tool in assessing applicants. As new data becomes available, revisions could be considered to the life experiences score to better predict offer of an interview, offer of admission, and post-admission academic success.

Reflective metacognition and objective structured clinical examination performance in introductory pharmacy practice experiences

Background and purposeObjective structured clinical examinations (OSCEs) allow for holistic evaluation of clinical competence, but integration of reflective practices on OSCE performance has not been well-studied. Educational activity and settingTo develop metacognitive skills, second-year pharmacy students completed a self-reflection assignment on a mock medication therapy management OSCE in an introductory pharmacy practice experience course. Students who completed the course during the prior year (no self-reflection assignment) served as the control. The study assessed the correlation of students' reflections of their OCSE abilities to OSCE performance and evaluated if student demographics influenced the correlation. FindingsThere were 107 students in the control group and 95 students in the intervention group. A higher proportion of students passed the overall OSCE in the control group (67.4%) compared to the intervention group (58.9%), but this was not statistically significant. Neither gender, age, nor Pharmacy College Admissions Test score had an effect on OSCE pass rates. Grade point average did have a significant effect on OSCE pass rates (P = .019). Overall, no statistically significant association was seen in summative OSCE performance between students who used structured, guided questions to reflect on their formative OSCE performance and those who did not (P > .05). SummaryDespite this study's results, faculty continue to embrace the concept of reflective practice. Further research is needed to assess the value and role of reflective practice in pharmacy education.

Building Representative Matched Samples With Multi-Valued Treatments in Large Observational Studies

ABSTRACTIn this article, we present a new way of matching in observational studies that overcomes three limitations of existing matching approaches. First, it directly balances covariates with multi-valued treatments without explicitly estimating the generalized propensity score. Second, it builds self-weighted matched samples that are representative of a target population by design. Third, it can handle large datasets, with hundreds of thousands of observations, in a couple of minutes. The key insights of this new approach to matching are balancing the treatment groups relative to a target population and positing a linear-sized mixed integer formulation of the matching problem. We formally show that this formulation is more effective than alternative quadratic-sized formulations, as its reduction in size does not affect its strength from the standpoint of its linear programming relaxation. We also show that this formulation can be used for matching with distributional covariate balance in polynomial time under certain assumptions on the covariates and that it can handle large datasets in practice even when the assumptions are not satisfied. This algorithmic characterization is key to handling large datasets. We illustrate this new approach to matching in both a simulation study and an observational study of the impact of an earthquake on educational attainment. With this approach, the results after matching can be visualized with simple and transparent graphical displays: while increasing levels of exposure to the earthquake have a negative impact on school attendance, there is no effect on college admission test scores. Supplementary materials for this article are available online.

Correlation of Medical College Admission Test Scores and Self-assessment Materials with the United States Medical Licensing Examination Step 1 Performance.

PurposeCandidates' performance on the United States Medical Licensing Examination (USMLE) Step 1 examination had been correlated with the Medical College Admission Test (MCAT). However, in 2015, a new MCAT format was released and its correlation with Step 1 remains to be fully analyzed. Preparation for Step 1 typically involves purchasing and perusing practice tests from the National Board of Medical Examiners (NBME) and UWorld; however, their predictive value to performance on Step 1 remains to be ascertained, especially with the release of five new NBME practice tests. Additionally, there is a need for accurately predicting Step 1 scores to self-evaluate study progress and reduce student anxiety.RationaleProgram directors rank USMLE Step 1 scores as the number one criterion in selecting interviewees for residency. Step 1 scores are more important than Step 2 scores, Dean’s letter, or other letters of recommendation in determining the overall ranking of a candidate after interviews.HypothesesThe authors hypothesized that the new MCAT scores correlated positively with Step 1 scores and that the new NBME practice tests were more predictive of performance on Step 1 as compared to old NBME tests. MethodsLinear regression analysis followed by either analysis of variance (ANOVA) or Student's t-tests were used to analyze 399 responses. Data obtained was used to update an existing Step 1 score predictor, which was then validated.ResultsA positive correlation between the MCAT (average score: 510.1 ± 6.3) and Step 1 scores (average score: 246.1 ± 14.2) was observed. While new NBME practice tests were more predictive of Step 1 scores than old NBME tests, UWorld test scores were the most predictive. Students who practiced with the new NBME practice tests scored significantly higher than students who did not use them. However, students using any of the UWorld practice tests did significantly better than students who practiced using only NBME practice tests but not UWorld practice tests. Ironically, NBME16,the second-most correlativetest to Step 1 performance, is no longer available for purchase. Overall, taking six or more practice tests significantly enhanced Step 1 scores; the optimal number of tests was found to be between six and nine. The predicted score by an updated Step 1 score predictor was within 3.8 points or 1.6% of the actual Step 1 score.ConclusionsWe believe this study will aid in the selection and purchase of appropriate self-assessment tests as preparatory material for the USMLE Step 1 examination. It will also introduce them to an existing Step 1 score predictor that will help determine their readiness for Step 1.

Predictors of success on the MCAT among post-baccalaureate pre-medicine students

Post-baccalaureate pre-medicine programs (PBPMP) provide prerequisite coursework for non-life science majors who aspire to become physicians. Students entering these programs generally do not have previous college-level exposure to the natural sciences. This pilot study was conducted to determine characteristics of scientifically naive, career changer, pre-medical students that may be used by PBPMP admissions committees. Statistical analyses were performed between Medical College Admission Test (MCAT) scores and student gender, Scholastic Aptitude Test (SAT) scores, undergraduate field of study, and undergraduate Grade Point Average (GPA). While relationships between certain subscores on the SAT and MCAT were found, data suggest that other non-quantitative metrics be considered as predictors of performance among PBPMP students.

Relationship between High School National Exam Scores and College Admission Test Scores in Indonesia

Relationship between High School National Exam Scores and College Admission Test Scores in Indonesia

The Consequences of Structural Racism on MCAT Scores and Medical School Admissions: The Past Is Prologue.

Those in medical education have a responsibility to prepare a physician workforce that can serve increasingly diverse communities, encourage healthy changes in patients, and advocate for the social changes needed to advance the health of all. The authors of this Perspective discuss many of the likely causes of the observed differences in mean Medical College Admission Test (MCAT) scores between students from groups well represented in medicine and those from groups underrepresented in medicine. The lower mean MCAT scores of underrepresented groups can present challenges to diversifying the physician workforce if medical schools only admit those applicants with the highest MCAT scores. The authors review the psychometric literature, which showed no evidence of bias in the exam, and note that the differences in mean MCAT scores between racial and ethnic groups are similar to those in other measures of academic achievement and performance on high-stakes tests.The authors then describe the ways in which structural racism in the United States has contributed to differences in achievement for underrepresented students compared with well-represented students. These differences are not due to differences in aptitude but to differences in opportunities. The authors describe the widespread consequences of structural racism on economic success, educational opportunity, and bias in the educational environment. They close with 3 recommendations for medical schools that may mitigate the consequences of structural racism while maintaining academic standards and admitting students likely to succeed. Adopting these recommendations may help the medical profession build the diverse physician workforce needed to serve communities today.

The Diversity and Success of Medical School Applicants With Scores in the Middle Third of the MCAT Score Scale.

Admissions officers assemble classes of medical students with different backgrounds and experiences who can contribute to their institutions' service, leadership, and research goals. While schools' local interests vary, they share a common goal: meeting the health needs of an increasingly diverse population. Despite the well-known benefits of diversity, the physician workforce does not yet reflect the nation's diversity by socioeconomic status, race/ethnicity, or other background characteristics.The authors reviewed the Medical College Admission Test (MCAT) scores and backgrounds of 2017 applicants, accepted applicants, and matriculants to U.S. MD-granting schools to explore avenues for increasing medical school class diversity. They found that schools that accepted more applicants with midrange MCAT scores had more diverse matriculating classes. Many schools admitting the most applicants with scores in the middle of the MCAT score scale were public, community-based, and primary care-focused institutions; those admitting the fewest of these applicants tended to be research-focused institutions and to report pressure to accept applicants with high MCAT scores to maintain or improve their national rankings.The authors argue that reexamining the use of MCAT scores in admissions provides an opportunity to diversify the physician workforce. Despite evidence that most students with midrange MCAT scores succeed in medical school, there is a tendency to overlook these applicants in favor of those with higher scores. To improve the health of all, the authors call for admitting more students with midrange MCAT scores and studying the learning environments that enable students with a wide range of MCAT scores to thrive.

National Council on Measurement in Education Position Statement on the Use of College Admissions Test Scores as Academic Indicators in State Accountability Systems

National Council on Measurement in Education Position Statement on the Use of College Admissions Test Scores as Academic Indicators in State Accountability Systems

Predictors of performance on the North Carolina Multistate Pharmacy Jurisprudence Examination

IntroductionTo evaluate an expanded set of variables and determine their association with Wingate University School of Pharmacy (WUSOP) graduates' scores on the North Carolina (NC) Multistate Pharmacy Jurisprudence Examination (MPJE). MethodsDemographic, pre-admission, pharmacy school, and licensing preparation variables were collected for WUSOP graduates who attempted the NC MPJE for the first time in 2017. Bivariate correlations between student-pharmacist characteristics and the MPJE score were performed. ResultsSeveral variables were found to have a statistically significant association with the NC MPJE score, including prerequisite grade point average, pharmacy coursework, Pharmacy Curriculum Outcomes Assessment performance, and completion of MPJE practice questions. There were no significant associations between NC MPJE scores and demographic characteristics, previous paid pharmacy employment, receipt of undergraduate degree, or Pharmacy College Admission Test scores. ConclusionThis exploratory analysis identified characteristics of WUSOP graduates that were associated with performance on the NC MPJE. The results of this study will be communicated with future graduates to enhance their success on the jurisprudence examination.

Institutional differences in USMLE Step 1 and 2 CK performance: Cross-sectional study of 89 US allopathic medical schools.

The United States Medical Licensing Examination (USMLE) Step 1 and Step 2 Clinical Knowledge (CK) are important for trainee medical knowledge assessment and licensure, medical school program assessment, and residency program applicant screening. Little is known about how USMLE performance varies between institutions. This observational study attempts to identify institutions with above-predicted USMLE performance, which may indicate educational programs successful at promoting students' medical knowledge. Self-reported institution-level data was tabulated from publicly available US News and World Report and Association of American Medical Colleges publications for 131 US allopathic medical schools from 2012-2014. Bivariate and multiple linear regression were performed. The primary outcome was institutional mean USMLE Step 1 and Step 2 CK scores outside a 95% prediction interval (≥2 standard deviations above or below predicted) based on multiple regression accounting for students' prior academic performance. Eighty-nine US medical schools (54 public, 35 private) reported complete USMLE scores over the three-year study period, representing over 39,000 examinees. Institutional mean grade point average (GPA) and Medical College Admission Test score (MCAT) achieved an adjusted R2 of 72% for Step 1 (standardized βMCAT 0.7, βGPA 0.2) and 41% for Step 2 CK (standardized βMCAT 0.5, βGPA 0.3) in multiple regression. Using this regression model, 5 institutions were identified with above-predicted institutional USMLE performance, while 3 institutions had below-predicted performance. This exploratory study identified several US allopathic medical schools with significant above- or below-predicted USMLE performance. Although limited by self-reported data, the findings raise questions about inter-institutional USMLE performance parity, and thus, educational parity. Additional work is needed to determine the etiology and robustness of the observed performance differences.

A Validity Argument in Support of the Use of College Admissions Test Scores for Federal Accountability

AbstractIn 2018, 26 states administered a college admissions test to all public school juniors. Nearly half of those states proposed to use those scores as their academic achievement indicators for federal accountability under the Every Student Succeeds Act (ESSA); many others are planning to use those scores for other accountability purposes. Accountability encompasses a number of different uses and subsumes a variety of claims. For states proposing to use summative tests for accountability, a validity argument needs to be developed, which entails delineating each specific use of test scores associated with accountability, identifying appropriate evidence, and offering a rebuttal to counterclaims. The aim of this article is to support states in developing a validity argument for use of college admission test scores for accountability by identifying claims that are applicable across states, along with summarizing existing evidence as it relates to each of these claims. As outlined by The Standards for Educational and Psychological Testing, multiple sources of evidence are used to address each claim. A series of threats to the validity argument, including weaker alignment with content standards and potential influences in narrowing teaching, are reviewed. Finally, the article contrasts validity evidence, primarily from research on the ACT, with regulatory requirements from ESSA. The Standards and guidance addressing the use of a “nationally recognized high school academic assessment” (Elementary and Secondary Education Act (ESEA), Negotiated Rulemaking Committee; Department of Education) are the primary sources for the organization of validity evidence.

Dynamic Measurement in Health Professions Education: Rationale, Application, and Possibilities.

Dynamic measurement modeling (DMM) is a psychometric paradigm that uses longitudinal data to estimate individual students' growth in measured skills over the course of an educational program (i.e., growth scores). DMM represents a more formal way of assessing learning progress across the health professions education continuum. In this article, the authors provide justification for this approach in health professions education and demonstrate its proof-of-concept use with three time points of United States Medical Licensing Examination Step exams to generate growth scores for 454 current and recent medical learners. The authors demonstrate that learners vary substantially on their growth scores, and those growth scores exhibit psychometric reliability. In addition, growth scores significantly and positively correlated with indicators of medical learner readiness (e.g., undergraduate grade point average and Medical College Admission Test scores). These growth scores were also capable of significantly and positively correlating with future ratings of clinical competencies during internship as assessed through a survey sent to their program directors at the end of the first postgraduate year (e.g., patient care, interpersonal skills). These preliminary findings of reliability and validity for DMM growth scores provide initial evidence for further investigation into the suitability of a dynamic measurement paradigm in health professions education.

Timing of Surgery and Internal Medicine Clerkships and Surgery Shelf Examination Scores

BackgroundMany medical students believe that third-year clerkship rotation sequence affects their success. We hypothesized that students who completed the internal medicine clerkship before the surgery clerkship received higher surgery shelf examination scores compared with the students who did not. Materials and methodsDeidentified academic data including preclinical data and National Board of Medical Examiners shelf examination scores for surgery for all third-year medical students at a single institution from 2012 to 2017 were analyzed. Students who did not complete all six core clerkships during the standard third-year time frame were excluded. Data were analyzed using 2-tailed t-tests and Z-scores. ResultsFour hundred and twenty four students were included in the study. Average undergraduate grade point average, Medical College Admission Test scores, and United States Medical Licensing Examination Step 1 scores showed no significant differences between groups. In aggregate, average shelf examination scores of students who completed the internal medicine clerkship before the surgery clerkship were significantly higher than those of students who did not. When the average shelf examination scores for the two groups were analyzed by individual rotation slot, no significant difference was found between the two groups. ConclusionsInitially, it appeared that students who completed the internal medicine clerkship before the surgery clerkship scored higher on their surgery shelf examinations. When the data were analyzed by individual rotation slot, we found no difference between the students who had already completed the internal medicine clerkship and those who had not. Experience over the year rather than completion of the internal medicine rotation was associated with higher surgery shelf examination scores.

Predicting success in pharmaceutical calculations

IntroductionUnderstanding preadmission predictors of success in pharmacy calculations, an important aspect of pharmacy practice, could impact admissions selection and advising practices. The objective of this study was to determine which student specific preadmission variables best predict success in a pharmaceutical calculations course. MethodsPreadmission data from 388 students who completed a one semester pharmaceutical calculations course between 2013 and 2016 were evaluated. This retrospective analysis was performed to determine which of the independent preadmission variables were positively correlated to the pharmaceutical calculations course grade. Fifteen preadmission variables, including demographics, grade point average (GPA), Pharmacy College Admission Test (PCAT) score, prior degrees, and number of pre-professional credit hours completed, were evaluated. Descriptive statistics were used for baseline characteristics and relative importance analysis was used to examine association between the dependent and independent variables. ResultsThe relative importance analysis revealed eight of the fifteen preadmission variables were significantly correlated with final grades in pharmaceutical calculations. Overall, 26.1% of the variance was explained by these variables. GPA (cumulative and science specific) and PCAT (quantitative and verbal) were the strongest indicators. ConclusionsPreadmission GPA and PCAT scores were the best predictors of success in a pharmaceutical calculations course. About one quarter of the variance was explained by the identified predictive variables, therefore other factors, not evaluated in this study, likely influence the success in pharmaceutical calculations and should be investigated to determine a more dependable set of predictors.

Impact of social media use on reading levels in third-year student pharmacists

IntroductionThe majority of Americans report using social media, but there is limited research describing impact of social media on academic performance and reading. Our objectives were to describe the association between social media use and reading levels of third-year student pharmacists (P3), describe the association between reading level and pharmacy school admissions data, and assess texts used in the curriculum for readability. MethodsThis was a prospective, cohort study. Reading level was determined by a standardized test. Social media data were collected via questionnaire. Admissions data were obtained from the admissions office. Readability of texts was assessed using readability software. ResultsEighty-nine student pharmacists completed the study. The average reading level was 16.4. Students reported using social media for an average of 126 min daily. Students reported using an average of four social media sites and spending 88 min weekly on extracurricular reading. Negligible linear correlations were found between reading level and time spent on social media (ρ = 0.063), number of sites used (ρ =0.062), and time spent on extracurricular reading (ρ= 0.130). A moderate correlation (ρ = 0.524) was found between reading level and Pharmacy College Admission Test (PCAT) score. The average readability of guidelines and textbook chapters were 18.1 ± 1.0 and 20.4 ± 0.3, respectively. ConclusionsIn P3 students, reading level was not associated with social media use. However, PCAT scores were positively associated with reading level. Furthermore, the readability of assigned texts exceeded the average reading level of the students.

Preparing incoming pharmacy students for a professional program: utilization of a practice lecture and comparative quizzes as a component of a bridging program.

Preparing incoming pharmacy students for a professional program: utilization of a practice lecture and comparative quizzes as a component of a bridging program.

Multivariable analysis of factors associated with USMLE scores across U.S. medical schools

BackgroundGauging medical education quality has always remained challenging. Many studies have examined predictors of standardized exam performance; however, data sets do not distinguish by institution or curriculum. Our objective is to present a summary of variables associated with the United States Medical Licensing Examination (USMLE) scores, and thus identify institutions (and therefore curriculums) which deviate from trend lines by producing higher USMLE scores despite having lower entrance grade point averages and medical college admissions test (MCAT) scores.MethodsData was obtained from U.S. News and World Report’s 2014 evaluation of allopathic U.S. medical schools. A univariate analysis was performed first for each variable using two sample t-test or Wilcoxon rank sum test for categorical variables, and Pearson or Spearman correlation coefficients for continuous variables. A multivariable linear regression model was developed to identify the factors contributing to USMLE scores. All statistical analyses were two-sided and performed using SAS software version 9.4 (SAS Institute Inc., Cary, NC).ResultsUnivariate analysis reveals a significant association between USMLE Step 1 and 2 scores with medical college admissions test scores, grade point averages, school type (private vs. public), full-time faculty-to-student ratio, National Institute of Health funds, residency director assessment score, peer assessment score, and class size. Of these nine variables, MCAT scores and Step 1 scores display the strongest correlation (corr = 0.72, P < .0001). Multivariable analysis also supports a significant association between MCAT scores and Step scores, meanwhile National Institute of Health funding size demonstrates a negative correlation with USMLE Step 2 scores. Although MCAT scores and National Institute of Health funds are significantly associated with USMLE performance, six outlier institutions were identified, producing higher USMLE scores than trend line predictions.ConclusionsOutlier institutions produce USMLE scores that do not follow expected trend lines. Their performance might be explainable by differences in curriculum. Having identified these institutions, their curriculums can be further studied to determine what factors enhance student learning.

Examining the Association of GPA and PCAT Scores on Objective Structured Clinical Examination Scores

Objective. To examine the association between certain demographic and admission measures and Objective Structured Clinical Examination (OSCE) performance in a cohort of pharmacy students. Methods. A retrospective review of demographic characteristics, admissions data [cumulative and science admission Grade Point Average (GPA), Pharmacy College Admissions Test (PCAT) scores], and OSCE scores was performed for the Class of 2017 at the University of Tennessee College of Pharmacy. Results. Female students scored significantly higher than male students on the Warfarin OSCE - Standardized Patient (SP) rated General Communication Skills and on the Warfarin OSCE - Faculty rated Patient Interviewing Skills. Age was significantly, inversely correlated with Warfarin OSCE - Faculty rated Therapeutic Knowledge score. Warfarin OSCE - SP rated General Communication Skills score was significantly, positively related to PCAT composite score and PCAT reading comprehension score. PCAT composite score was significantly, inversely correlated to Warfarin OSCE - Faculty rated Patient Interviewing Skills score. Warfarin OSCE - Faculty rated General Communication Skills score was significantly, positively related to cumulative admission GPA and admission science GPA. Conclusion. Eight statistically significant correlations were found between demographic and admissions measures and specific OSCE scores. Regression models were significant but explained a low percentage of the variance in OSCE scores, suggesting other factors not included in the study have a greater effect on scores. Such factors may include knowledge acquired in pharmacy school courses such as the therapeutics course series.

Longitudinal Assessment of Medical Student Emotional Intelligence Over Preclinical Training.

Given the importance of emotional intelligence (EI) in physician leadership and success, the possible decline of EI over the course of training is a concern in medical education. To assess the EI of osteopathic medical students as they progress through the preclinical years of medical school, to discover the course of EI over time, and to identify specific core EI competencies to be enhanced or coached. This exploratory, longitudinal study used survey data to assess the course and competencies of EI in the incoming 2019 class of medical students at a college of osteopathic medicine. Changes in EI overall scores and the 8 core competency scale scores were assessed, and additional demographic data were collected for use in analysis. Methods of analysis included paired samples t test, independent samples t test, Pearson product moment correlation, and multiple regression analysis. Statistical significance was defined as P<.05. Sixty-two participants completed the Six Seconds Emotional Intelligence Assessment survey at orientation and again after their second year. Overall EI scores declined over the course of the preclinical training for men and women (t61=4.24, P<.001), although no differences were noted by gender when independent-samples t tests were run. However, 2 of the 8 competency scales of the composite EI scores did not decline over time-Enhanced Emotional Literacy and Pursue Noble Goals. A weak negative correlation was found between Medical College Admission Test scores and the scale of Apply Consequential Thinking (r=-0.3, P<.05). A weak positive correlation was found between grade point average and intrinsic motivation (r=0.3, P<.05) and optimism (r=0.3, P<.05), and a moderate positive correlation existed between preboard examination scores and intrinsic motivation (r=0.5, P<.001) and optimism (r=0.4, P<.01). None of the chosen independent variables (academic grades, test scores, age, or gender) contributed to overall EI scores. During the preclinical training, osteopathic medical students' overall EI scores, as well as 6 of the 8 core competency scales, declined. Therefore, EI coaching is warranted to address specific core competencies needed to build, enhance, and prevent the decline of EI through preclinical training.