College Admission Test Research Articles

College Admissions Tests and LD and ADHD Documentation Guidelines: Consistency With Emerging Legal Guidance

Testing agencies request documentation to verify a test-taker’s disability status under the Americans With Disabilities Act of 2008 and Section 504 of the Rehabilitation Act of 1973. A number of recent legal developments, culminating in technical assistance from the U.S. Department of Justice, suggest changes in enforcement of relevant laws are imminent. This article reviews the legal developments and presents results of a survey of the learning disability and Attention-Deficit/Hyperactivity Disorder documentation guidelines of 10 standardized tests commonly used to aid admissions decisions for undergraduate and graduate programs. Consistent with the recent guidance, virtually all guidelines requested documentation from a qualified evaluator, a diagnosis, and accommodation recommendations and rationale. In contrast, agencies were less likely to request the information necessary to grant accommodations based on history of accommodation use. Few agencies had policies allowing submission of reduced amounts of documentation for students with lengthy histories of disabling conditions. Finally, guidelines frequently neglected to provide the guidance that evaluators need to generate useful documentation. Benefits and limitations of the recent legal guidance are discussed, and recommendations for testing and enforcement agencies, as well as secondary schools, are provided.

Examining the Impact of Covariates on Anchor Tests to Ascertain Quality Over Time in a College Admissions Test

We propose a comprehensive procedure for the implementation of a quality control process of anchor tests for a college admissions test with multiple consecutive administrations. We propose to examine the anchor tests and their items in connection with covariates to investigate if there was any unusual behavior in the anchor test results over time and if the test results differed for different groups of test takers. Descriptive statistics, ANOVA, and linear mixed effect models were used to examine the impact of covariates on the anchor test over time. Descriptive statistics and logistic regression were used to examine the quality of each anchor item over time. It is concluded that examining the impact of the test takers' covariates at different administrations can help in the process of providing fair tests for all test takers. The results are discussed with recommendations how it can be used in other large testing programs.

Multiple mini-interview predictive validity for performance on a pharmacy licensing examination.

Predictive validity studies on the use of the multiple mini-interview (MMI) have been primarily in medicine. This study sought to determine the predictive validity of the MMI for performance within a pharmacy programme and on the Pharmacy Examining Board of Canada (PEBC) Qualifying Examination for licensure, and to compare the predictive validity of the MMI with that of pre-pharmacy grade point average (GPA) and Pharmacy College Admission Test (PCAT) score. Admissions data for 223 graduates of the pharmacy programme at the University of Toronto were matched to programme and licensure outcome measures. Multiple linear regression assessed the predictive ability of the MMI, pre-pharmacy GPA, PCAT and covariates for performance in final-year experiential rotations, cumulative GPA (cGPA) and PEBC-MCQ (multiple-choice question examination) and PEBC-OSCE (objective structured clinical examination) overall and subcomponent scores. The PCAT, pre-pharmacy GPA and age significantly predicted the PEBC-MCQ overall score. The MMI was the only significant predictor of overall score on the PEBC-OSCE (β = 0.17, p = 0.02); it also predicted communication and performance subscores. Scores on the PCAT and female gender predicted the communication subscore. Pre-pharmacy GPA, age and female gender significantly predicted cGPA. The MMI was the only significant predictor of institutional/ambulatory rotation score (β = 0.26, p = 0.00). The MMI, designed to measure non-academic attributes including communication, motivation and problem-solving skills, was the only admissions tool with significant predictive validity for performance on the PEBC-OSCE national pharmacy certification examination and in an institutional/ambulatory rotation. These findings, from a single cohort of undergraduates, provide the first report of the predictive validity of the MMI for performance on pharmacy licensure examinations and thereby strengthen the evidence for its use in health professions selection. Prior university academic performance significantly predicted cGPA and performance on the PEBC-MCQ. Performance on the PCAT also predicted PEBC-MCQ results.

Master Students’ cGPA Is a Good Predictor of Success in Medical School

The Medical College Admission Test (MCAT) and undergraduate Grade Point Average (GPA) are used as primary criteria for admission to medical schools across the country. Prospective students are often considered uncompetitive when they do not meet the minimum MCAT score required even when they have performed well in a post-baccalaureate or a master program. Touro University Nevada (TUN) College of Osteopathic Medicine offers a rigorous Master of Health Sciences (MHS) program where students both take the majority of fall semester courses and are assessed with the first year medical students. In order to determine the best admission criteria for MHS students into medical school, this study aims to determine whether there are correlations between the MHS cumulative GPA (cGPA), the MCAT score, the preclinical years’ cGPA, and the scores on the Comprehensive Osteopathic Medical Licensing Examinations (COMLEX) of MHS students. The results indicate that there are strong correlations between these students’ MHS cGPAs and their preclinical years’ cGPAs and their COMLEX scores. Conversely, there are no significant correlations between their MCAT scores and their medical school performance indicators. Consequently, MHS cGPA appears to be a more reliable predictor of students’ performance in our medical school. It is plausible that the MCAT scores should be considered as a secondary criterion for admission of MHS students who perform especially well in the program. For these students, the MHS cGPA appears to be a better-suited primary criterion for admission to medical school.

Academic Performance on First-Year Medical School Exams: How Well Does It Predict Later Performance on Knowledge-Based and Clinical Assessments?

ABSTRACTNumber of appearances in the bottom quartile of 1st-year medical school exams were used to represent the extent to which students were having academic difficulties. Medical educators have long expressed a desire to have indicators of medical student performance that have strong predictive validity. Predictors traditionally used fell into 4 general categories: demographic (e.g., gender), other background factors (e.g., college major), performance/aptitude (e.g., medical college admission test scores), and noncognitive factors (e.g., curiosity). These factors, however, have an inconsistent record of predicting student performance. In comparison to traditional predictive factors, we sought to determine the extent to which academic performance in the 1st-year of medical school, as measured by examination performance in the bottom quartile of the class in 7 required courses, predicted later performance on a variety of assessments, both knowledge based (e.g., United States Medical Licensing Examination Step 1 and Step IICK) and clinical skills based (e.g., clerkship grades and objective structured clinical exam performance). Of all predictors measured, number of appearances in the bottom quartile in Year 1 was the most strongly related to performance in knowledge-based assessments, as well as clinically related outcomes, and, for each outcome, bottom-quartile performance accounted for additional variance beyond that of the traditional predictors. Low academic performance in the 1st year of medical school is a meaningful risk factor with both predictive validity and predictive utility for low performance later in medical school. The question remains as to how we can incorporate this indicator into a system of formative assessment that effectively addresses the challenges of medical students once they have been identified.

Factors that may influence the score in the College Admission Test in China

<b>Introduction: </b> Many factors may influence the score of the College Admission Test. The purpose of this study was to assess the factors that may predict students’ scores in the College Admission Test in China. <br /> <b>Material and methods: </b> We sent out 1000 questionnaires which consisted of the following items: objective data on demographic factors, their scores of the test, the year they attended the test, whether they took a set of “art” subjects or a set of “science” subjects, the monthly salary of the whole family, the highest academic degree obtained by the father and mother, the history of past operations and the name and the times of operations undergone, as well as the type of anesthesia received by those who took the College Admission Test. <br /> <b>Results</b>: Univariate analyses showed that age, gender, household income, parental education, and anesthesia history may influence College Admission Test scores. These variables were subsequently entered into the multivariate linear regression model. Students with a history of local anesthesia are predicted to have a score 8.1 points lower than the average (–8.1, 95% CI: –15, –1) and this effect is statistically significant (p = 0.025). Students with a history of general anesthesia are also predicted to have lower scores (–2.8, 95% CI: –16, 10.2) than the average, but the difference is not statistically significant. <br /> <b>Conclusions</b>: The present findings suggest that the scores in the College Admission Test may be associated with age, gender, father’s education and the use of local anesthetics. Ever receiving local anesthesia had a negative impact on College Admission Test scores.

Social Capital and College-Going Culture in High Schools: the Effects of College Expectations and College Talk on Students' Postsecondary Attendance

Using the Education Longitudinal Study of 2002, we examined the probability of students attending college by taking into account their 10th- and 12th-grade exposure to college expectations from and college-related interactions with school staff. Our results suggest that college expectations and college-related interactions with their school counselors, teachers, and coaches had a modest impact on students' odds of enrolling in college, considering the large impact of taking advanced courses and completing college admissions tests.

Factors Associated with Performance in an Internal Medicine Clerkship

The purpose of this retrospective study was to examine the relationship between demographic and educational variables and student performance on an internal medicine (IM) clerkship in order to target areas for educational intervention and potential early remediation. This study examined data associated with third-year medical student performance (N = 505) during the IM clerkship at Baylor Scott & White, Temple/Texas A&M Health Science Center College of Medicine from 2005 to 2011. Multiple regression analysis (N = 341) showed that a model containing the following variables was significantly associated with scores on the National Board of Medical Examiners (NBME) subject exam, accounting for 46.5% of the variance: Objective Structured Clinical Exam (OSCE), Medical College Admissions Test (MCAT), US Medical Licensing Exam (USMLE) Step 1, second-year grade point average (GPA), and clinical evaluation. A model containing USMLE Step 1, clinical evaluation, and NBME was significantly associated with OSCE score, accounting for 30% of the variance. Additionally, a model containing age, MCAT score, undergraduate GPA, NBME subject exam score, and OSCE was significantly associated with clinical evaluation score, accounting for 22% of score variance. Age accounted for the most unique variance in clinical evaluation score. Gender and IM interest group were not significantly associated with any outcome variable. In conclusion, in contrast to previous studies in the field, we did not find a significant association between undergraduate GPA and NBME score. Our findings supply further evidence that the OSCE, typically believed to be a clinical performance exam, actually assesses a broader set of domains. Interest group membership did not confer any academic benefit to medical students in IM clerkships in our study.

Components of an Evidence-Based Analytic Rubric for Use in Medical School Admissions.

Attrition from medical school remains a serious cause of concern for the medical education community. Thus, there is a need to improve our ability to select only those candidates who will succeed at medical school from many highly qualified and motivated applicants. This can be achieved, in part, by reducing the reliance on cognitive factors and increasing the use of noncognitive character traits in high-stakes admissions decisions. Herein we describe an analytic rubric that combines research-derived predictors of medical school success to generate a composite score for use in admissions decisions. The analytic rubric as described herein represents a significant step toward evidenced-based admissions that will facilitate a more consistent and transparent qualitative evaluation of medical school applicants beyond their grades and Medical College Admissions Test scores and contribute to a redesigned and improved admissions process.

Development of an interdisciplinary pre-matriculation program designed to promote medical students’ self efficacy

ABSTRACTBackground and objectives: A four-week interdisciplinary pre-matriculation program for Native American and rural medical students was created and its impact on students’ transition to medical school was assessed. The program extends the goals of many pre-matriculation programs by aiming to increase not only students’ understanding of basic science knowledge, but also to build student self-efficacy through practice with medical school curricular elements while developing their academic support networks.Design: A mixed method evaluation was used to determine whether the goals of the program were achieved (n = 22). Student knowledge gains and retention of the microbiology content were assessed using a microbiology concept inventory. Students participated in focus groups to identify the benefits of participating in the program as well as the key components of the program that benefitted the students.Results: Program participants showed retention of microbiology content and increased confidence about the overall medical school experience after participating in the summer program.Conclusions: By nurturing self-efficacy, participation in a pre-matriculation program supported medical students from Native American and rural backgrounds during their transition to medical school.Abbreviations: CAIMH: Center of American Indian and Minority Health; MCAT: Medical College Admission Test; PBL: Problem based learning; UM MSD: University of Minnesota Medical School Duluth

Talento académico, vulnerabilidad escolar y resultados en la prueba de selección universitaria

Relation between giftedness, cognitive abilities, participation in a gifted program, school vulnerability and the Chilean university admission test scores was studied in a sample of 1303 students who applied to PentaUC Program between 2001 and 2004, and took the Chilean University Admission Test (PSU) between 2007 and 2010. Results show that students who complete a gifted program obtain better PSU scores than those who drop out. Vulnerability of academic contexts and the type of content evaluated in PSU, appeared as variables that modulate both the effect of the program and the effect of giftedness on PSU scores. High cognitive abilities seem to have no direct influence on PSU scores when those scores are at the upper end of the scale.

One picture or a thousand words? Influence of question length and illustration support on the success and skip rates on online tests

The growing popularity of automatically graded online tests, either as an evaluation or self-assessment tool in online or blended education, demands a review of how these questions are designed and delivered to their intended audience. This paper analyzes the results of over 20,000 pre-university mock online quizzes designed to train the students for the Spanish university admission test (known as “Pruebas de Acceso a la Universidad” or “Selectividad”) in the technical drawing subject, corresponding to the June and September intakes of 2009 and 2015. The influence of two key aspects on the questions success and skip rates is assessed: (a) the presence or absence of illustration support and (b) the length of the question as a proxy of reading comprehension difficulty. The results support that the presence of an accompanying illustration in the questions result in fewer skipped questions and mode successful answers, while the length of the question has the opposite effect. The performance difference in the 6-year span is also discussed, showing a slight decline over time in the pass rates while the skip rates remain stable. When comparing both two intakes, corresponding to different academic profiles of students that passed the June exam and those who did not, the success ratio is unsurprisingly lower for the students in the second intake. These findings should help improving the design of online quizzes, including more visual content and/or rephrasing the questions to be more concise, to fit the requirements of students educated in a more visual environment of multimedia technologies.

Licensure Examination Performance of BSEd-Biological and Physical Science Graduates in a State University in Northern Philippines

This study analyzed the licensure examination for teachers (LET) performance of the Bachelor of Secondary Education (BSEd)-Biological and Physical Science Graduates in the University of Northern Philippines. Utilizing the descriptive correlational method, it involved the BSEd graduates major in Physical and Biological Sciences, School Years 2012-2013 to 2014-2015. The study employed documentary analysis of records of the graduates. Mean, analysis of variance, and multiple regression were used to analyze the data. Results showed that most of the respondents had graduated in public high schools and a great majority of them attended LET review. Their performance in the College Admission Test (CAT) is almost equally distributed from very good, good and passing, and nearly all of them obtained a passing mark in the Teaching Aptitude Test (TAT). The overall LET performance of the respondents is above the National Passing Percentage in general education, professional and major subjects. Respondents’ LET performance in 2015 is significantly higher than their performance in 2014. The performance of the LET takers in general education is higher in 2015 than in 2014. The respondents with high achievement in professional education subjects attended LET review and with high College Admission Test have better overall LET performance along general education and professional education components. The respondents with higher achievement in general education subjects have better performance in the specialization aspects. The institution may implement higher cut-off score in the College Admission Test. The college may conduct activities related to the licensure examination to prepare thoroughly the LET takers.

Scylla and Charybdis: The MCAT, USMLE, and Degrees of Freedom in Undergraduate Medical Education.

In recent years, medical educators have been making meaningful attempts to rethink how premedical students are prepared for medical school, and how medical students are prepared for residency. Among the many challenges to redesigning premedical and medical school curricula, one that stands out is the constraint imposed by our current methods of assessing aptitude, particularly our use of the Medical College Admissions Test (MCAT) and the United States Medical Licensing Examination (USMLE). For much of the past century, medical school and residency admissions committees have relied heavily on MCAT and USMLE scores to evaluate and rank candidates to their programs. These high-stakes exams determine to a large extent what is taught, and what is stressed, in preparation for and during medical school-despite the fact that scores have limited ability to predict future success in clinical medicine or biomedical research. Additionally, evidence indicates that students from disadvantaged and minority backgrounds do not fare as well on these exams and, as a result, may be disproportionately excluded from the medical profession. While medical school admissions committees have made limited incremental gains in holistic review, residency programs appear to be increasingly focused on USMLE Step scores and veering away from the spirit of holistic review. The authors propose that substantive change will remain slow in coming unless members of the medical education community radically rethink how we report scores from these exams, and how we use them in our selection of future medical students and residents.

The Influence of MCAT and GPA Preadmission Academic Metrics on Interview Scores

Medical school admissions interviews are used to assess applicants’ nonacademic characteristics as advocated by the Association of American Medical Colleges’ Advancing Holistic Review Initiative. The objective of this study is to determine whether academic metrics continue to significantly influence interviewers’ scores in holistic processes by blinding interviewers to applicants’ undergraduate grade point averages (uGPA) and Medical College Admission Test (MCAT). This study examines academic and demographic predictors of interview scores for two applicant cohorts at the University of Michigan Medical School. In 2012, interviewers were provided applicants’ uGPA and MCAT scores; in 2013, these academic metrics were withheld from interviewers’ files. Hierarchical regression analysis was conducted to examine the influence of academic and demographic variables on overall cohort interview scores. When interviewers were provided uGPA and MCAT scores, academic metrics explained more variation in interview scores (7.9%) than when interviewers were blinded to these metrics (4.1%). Further analysis showed a statistically significant interaction between cohort and uGPA, indicating that the association between uGPA and interview scores was significantly stronger for the 2012 unblinded cohort compared to the 2013 blinded cohort (β = .573, P < .05). By contrast, MCAT scores had no interactive effects on interviewer scores. While MCAT scores accounted for some variation in interview scores for both cohorts, only access to uGPA significantly influenced interviewers’ scores when looking at interaction effects. Withholding academic metrics from interviewers’ files may promote assessment of nonacademic characteristics independently from academic metrics.

Enrollment Management in Medical School Admissions: A Novel Evidence-Based Approach at One Institution.

In higher education, enrollment management has been developed to accurately predict the likelihood of enrollment of admitted students. This allows evidence to dictate numbers of interviews scheduled, offers of admission, and financial aid package distribution. The applicability of enrollment management techniques for use in medical education was tested through creation of a predictive enrollment model at the University of Michigan Medical School (U-M). U-M and American Medical College Application Service data (2006-2014) were combined to create a database including applicant demographics, academic application scores, institutional financial aid offer, and choice of school attended. Binomial logistic regression and multinomial logistic regression models were estimated in order to study factors related to enrollment at the local institution versus elsewhere and to groupings of competing peer institutions. A predictive analytic "dashboard" was created for practical use. Both models were significant at P < .001 and had similar predictive performance. In the binomial model female, underrepresented minority students, grade point average, Medical College Admission Test score, admissions committee desirability score, and most individual financial aid offers were significant (P < .05). The significant covariates were similar in the multinomial model (excluding female) and provided separate likelihoods of students enrolling at different institutional types. An enrollment-management-based approach would allow medical schools to better manage the number of students they admit and target recruitment efforts to improve their likelihood of success. It also performs a key institutional research function for understanding failed recruitment of highly desirable candidates.

Predictors of student performance on the Pharmacy Curriculum Outcomes Assessment at a new school of pharmacy using admissions and demographic data

ObjectivesTo characterize student performance on the Pharmacy Curriculum Outcomes Assessment (PCOA) and to determine the significance of specific admissions criteria and pharmacy school performance to predict student performance on the PCOA during the first through third professional years. MethodsMultivariate linear regression models were developed to study the relationships between various independent variables and studentsʼ PCOA total scores during the first through third professional years. ResultsTo date, four cohorts have successfully taken the PCOA examination. Results indicate that the Pharmacy College Admissions Test (PCAT), the Health Science Reasoning Test (HSRT), and cumulative pharmacy grade point average were the only consistent significant predictors of higher PCOA total scores across all students who have taken the exam at our school of pharmacy. ConclusionThe school should examine and clarify the role of PCOA within its curricular assessment program. Results suggest that certain admissions criteria and performance in pharmacy school are associated with higher PCOA scores.

Differential prediction generalization in college admissions testing.

We introduce the concept of differential prediction generalization in the context of college admissions testing. Specifically, we assess the extent to which predicted first-year college grade point average (GPA) based on high-school grade point average (HSGPA) and SAT scores depends on a student’s ethnicity and gender and whether this difference varies across samples. We compared 257,336 female and 220,433 male students across 339 samples, 29,734 Black and 304,372 White students across 247 samples, and 35,681 Hispanic and 308,818 White students across 264 samples collected from 176 colleges and universities between the years 2006 and 2008. Overall, results show a lack of differential prediction generalization because variability remains after accounting for methodological and statistical artifacts including sample size, range restriction, proportion of students across ethnicity- and gender-based subgroups, subgroup mean differences on the predictors (i.e., HSGPA, SAT-Critical Reading, SAT-Math, and SAT-Writing), and SDs for the predictors. We offer an agenda for future research aimed at understanding several contextual reasons for a lack of differential prediction generalization based on ethnicity and gender. Results from such research will likely lead to a better understanding of the reasons for differential prediction and interventions aimed at reducing or eliminating it when it exists. (PsycINFO Database Record (c) 2016 APA, all rights reserved)

An Evidence-Supported Medical Laboratory Science Program Admissions Selection Process

After a recent enrollment expansion, a Medical Laboratory Science program experienced a higher than desired student attrition rate due to a number of academic and non-academic student readiness factors. In an attempt to address retention and other issues, the program completed a curriculum update and revision as well as a conversion to a hybrid or flipped classroom delivery model, but in spite of an overall improvement in student learning outcomes, the program still experienced a high level of student attrition. Program faculty then developed and implemented a two stage holistic admissions selection process, which included an interview and skills test, in an attempt to assess candidate background knowledge and abilities in an equitable manner. Comparison of student factors associated with on-time successful graduation, probation (delayed graduation), or dismissal from the program indicated that the science and prerequisite science grade point averages were significantly higher for students who graduated on-time compared to delayed or non-successful (dismissed) students. Review of applicants9 performance in the interview and skills test showed significant differences for multiple factors for students who graduated on-time from the program compared to delayed (probation) and non-successful (dismissed) students. Continuing reviews of program retention rates are needed, however the attrition rate for the next cohort dropped from 24% to 4% when the program focused the selection process on factors shown to be associated with successful graduation. ABBREVIATIONS:MCAT - Medical College Admission Test, ACT – American College Test, SAT – Scholastic Aptitude Test, GPA – Grade Point Average, AHPAT – Allied Health Professions Admission Test, MLS – Medical Laboratory Sciences, ANOVA - Analysis of variance, sGPA - Science GPA, preGPA – Prerequisite science course GPA, TOEFL - Test of English as a Foreign Language, PPB – Purdue Pegboard

Predicting Freshman Grade Point Average From College Admissions Test Scores and State High School Test Scores

The current focus on assessing “college and career readiness” raises an empirical question: How do high school tests compare with college admissions tests in predicting performance in college? We explored this using data from the City University of New York and public colleges in Kentucky. These two systems differ in the choice of college admissions test, the stakes for students on the high school test, and demographics. We predicted freshman grade point average (FGPA) from high school GPA and both college admissions and high school tests in mathematics and English. In both systems, the choice of tests had only trivial effects on the aggregate prediction of FGPA. Adding either test to an equation that included the other had only trivial effects on prediction. Although the findings suggest that the choice of test might advantage or disadvantage different students, it had no substantial effect on the over- and underprediction of FGPA for students classified by race-ethnicity or poverty.