Admissions Testing Program Research Articles

Universal admission laboratory screening for severe acute respiratory coronavirus virus 2 (SARS-CoV-2) asymptomatic infection across a large health system

Admission laboratory screening for asymptomatic coronavirus disease 2019 (COVID-19) has been utilized to mitigate healthcare-associated severe acute respiratory coronavirus virus 2 (SARS-CoV-2) transmission. An understanding of the impact of such testing across a variety of patient populations is needed. SARS-CoV-2 nucleic acid amplification admission testing results for all asymptomatic patients across 4 distinct inpatient facilities between April 20, 2020, and June 14, 2021, were analyzed. Positivity rates and the number needed to test (NNT) to identify 1 asymptomatic infected patient were calculated. Admission results were compared to COVID-19 community incidence rates for the system's surrounding metropolitan service area. Using a national survey of hospital epidemiologists, a clinically meaningful NNT of 1:100 was identified. In total, 51,187 tests were collected (positivity rate, 1.8%). During periods of high transmission, the NNT met the clinically relevant threshold in all populations. The NNT approached or met the threshold for most locations during periods of lower transmission. For all transmission levels, the NNT for fully vaccinated patients did not meet the threshold. Implementing an asymptomatic patient admission testing program can provide clinically relevant data based on the NNT, even during periods of lower transmission and among different patient populations. Limiting admission testing to non-fully vaccinated patients during periods of lower transmission may be a strategy to address resource concerns around this practice. Although the impact of such testing on healthcare-associated COVID-19 among patients and healthcare workers could not be clearly determined, these data provide important information as facilities weigh the costs and benefits of such testing.

Analysis of Universal admission laboratory screening for SARS-CoV-2 asymptomatic infection across a large health system

Background: Admission laboratory screening for asymptomatic COVID-19 has been utilized to mitigate healthcare-associated SARS-CoV-2 transmission. A better understanding of the impact of such testing across a variety of patient populations is needed. Methods: Beginning April 2020, every patient admitted within an academic healthcare system underwent SARS-CoV-2 PCR testing upon admission. Between April 20, 2020 through June 14, 2021, results were analyzed in asymptomatic patients across 4 inpatient facilities: a tertiary-care adult hospital, a free-standing pediatric hospital, a community-based hospital, and a behavioral health hospital. Positivity rates and the number needed to test (NNT) to identify 1 asymptomatic infected patient were calculated overall, by hospital type, by patient vaccination status, and by CDC-defined levels of community transmission. Weekly community incidence rates of COVID-19 for the system’s metropolitan service area (8 central Tennessee counties) were obtained from Tennessee Department of Health records. Weekly COVID-19 incidence rates per 100,000 people were calculated using US Census Bureau data. Using a national survey of hospital epidemiologists, a clinically meaningful NNT was identified (ie, 1 positive patient per 100 patients tested). A crude admission testing cost (covering testing supplies, reagents, and lab personnel costs) was obtained from operational data ($50 per test) to assess testing utility. Results: In total, 51,187 tests were collected during the study period with a positivity rate of 1.8%. No periods of low transmission were observed (Table 1). During high transmission periods, the NNT met the clinically relevant threshold in all populations. In addition, the NNT approached or met the 1:100 threshold for most locations during periods of less transmission, suggesting continued benefit even as infection rates decline. In all transmission periods, the NNT for non–fully vaccinated patients met the clinically meaningful threshold, in contrast to testing of fully vaccinated patients (Table 2). Discussion: Implementing an asymptomatic patient admission testing program can provide clinically relevant value based on the NNT, even during lower periods of transmission and in different patient populations. Limiting admission testing to non–fully vaccinated patients during periods of lower transmission may be a strategy to address cost and resource concerns around this practice. Further investigations into the impact of booster vaccination and newer SARS-CoV-2 variants on admission testing programs are also necessary. Although the impact of such testing on healthcare-associated COVID-19 among patients and healthcare workers could not be clearly determined, these data provide important information as facilities weigh the costs and benefits of such testing.Funding: NoneDisclosures: None

Consider Propensity Scores to Compare Treatments

The underlying question when comparing treatments is usually whether an individual would do better with treatment X than they would with treatment Y. But there are often practical and theoretical problems in giving people both treatments and comparing the data. This paper presents the use of propensity score matching as a methodology that can be used to compare the effectiveness of different treatments. The method is applied to answer two questions: (1) “Should examinees take a college admissions test near or a few years after graduation?” and (2) “Do accommodated students receive an unfair advantage?” Data from a large admission testing program is used.

Gender-related differences in cognitive abilities: Evidence from a medical school admissions testing program

In an investigation of gender-related differences in cognitive ability factors, analyses were undertaken of a series of administrations over a 9-yr period of a comprehensive test battery used to select medical school applicants in West Germany. Fifteen correlation matrices based on a total of 96,968 males and 90,142 females were factor analysed. Three factors were extracted in every case and rotated to an orthogonal simple structure using the Varimax procedure. In every instance, the three factors were identified as reasoning, perceptual speed, and memory with congruence coefficients across administration ranging from 0.89 to 0.99. Highly similar factors were also identified when the data of males and females were factored separately. In all 15 analyses, males scored higher on the reasoning factor than did females, and females scored higher than males on the memory factor, in each case about one-half of a standard deviation. Clear changes over the years were not in evidence, except for a tendency for the female advantage in memory to decline.

A COMPARISON OF THE PROPERTIES OF IRT PARAMETER ESTIMATES USING TWO DIFFERENT CALIBRATION DESIGNS

ABSTRACTThe purpose of this study was to compare two different methods of obtaining 3PL IRT pretest item parameter estimates for the Graduate Management Admissions Testing Program. The first method consisted of calibrating pretest and operational items simultaneously in a LOGIST run, that is, a concurrent calibration design. The second approach entailed analyzing the pretest items separately from the operational items holding examinee ability scores constant from a previous operational items run, that is, using a two‐stage calibration design. Results show that the means of the item difficulty (b‐parameter) estimates were very similar, regardless of the method employed. However, the higher b‐parameter values using the two‐stage calibration run method (i.e. holding ability fixed excluding the studied items from the criterion) were slightly overestimated and the lower b‐parameter values were slightly underestimated. The a‐parameters were consistently underestimated using the two‐stage estimation procedure. Finally, the slopes of the item‐ability regressions using a concurrent calibration (including the studied items) are steeper for nearly all of the items. These preliminary results are consistent with those reported in past studies (Stocking & Eignor, 1986) and suggest that non‐operational (pretest) items should be calibrated concurrently with operational items for item banking purposes.

CASE STUDIES IN COMPUTER ADAPTIVE TEST DESIGN THROUGH SIMULATION1,2

ABSTRACTThe extensive computer simulation work done in developing the computer adaptive versions of the Graduate Record Examinations (GRE) Board General Test and the College Board Admissions Testing Program (ATP) SAT is described in this report. Both the GRE General and SAT computer adaptive tests (CATs), which are fixed length in nature, were developed from pools of items that were calibrated using the three‐parameter logistic IRT model and item selection was based on the recently developed weighted deviations algorithm (see Swanson and Stocking, 1992), which simultaneously deals with content, statistical, and other constraints in the item selection process. For the GRE General CATs (Verbal, Quantitative, and Analytical), item exposure was controlled by using an extension of an approach originally developed by Sympson and Hetter (1988). For the SAT CATs (Verbal and Mathematical), item exposure was controlled by using a less complex randomization approach. Lengths of the CATs were determined so that CAT reliabilities matched or exceeded comparable full length paper‐and‐pencil test reliabilities.

DERIVING COMPARABLE SCORES FOR COMPUTER ADAPTIVE AND CONVENTIONAL TESTS: AN EXAMPLE USING THE SAT1,2

ABSTRACTProcedures used to establish the comparability of scores derived from the College Board Admissions Testing Program (ATP) computer adaptive SAT prototype and the paper‐and‐pencil SAT are described in this report. Both the prototype, which is made up of Verbal and Mathematical computer adaptive tests (CATs), and a form of the paper‐and‐pencil test were administered to just greater than 500 examinees using a random groups counterbalanced design. Both linear and equipercentile procedures were used for equating in each of the separate testing orders (paper‐and‐pencil then CAT or CAT then paper‐and‐pencil). Data were not pooled across the orders because the groups were not randomly equivalent due to administrative problems. The linear procedure was chosen for each test (Verbal or Mathematical) for each order, and results from the two orders were averaged. The final Verbal and Mathematical CAT conversions were quite similar to the paper‐and‐pencil conversions, although the two conversions for Verbal and two conversions for Mathematical did differ by as much as 20 scaled score points in certain regions of the scale.

A STUDY OF GENDER AND PERFORMANCE ON ADVANCED PLACEMENT HISTORY EXAMINATIONS

ABSTRACTSeveral studies have shown that, on average, women perform slightly better than men on free‐response tests, while men perform slightly better on multiple‐choice tests, Studies of the College Board Advanced Placement (AP) Examinations have revealed a similar phenomenon. For almost all AP Examinations, men average better on both parts of the tests, but sex differences on the free‐response parts are almost always less, and for some tests they are nonsignificant.Two AP Examinations, U.S. History and European History, were selected for study because sex differences on the free‐response parts were nonsignificant while sex differences on the multiple‐choice parts were large. Random samples of free‐response booklets were drawn from the 1986 administrations of both examinations. A number of ratings and analyses were made of the responses: English composition quality, historical content, responsiveness, factual errors, handwriting quality, neatness, and number of words written. All variables were then used to predict the free‐response scores. Several significant predictors were observed: the AP multiple‐choice score, historical content, English composition quality, and the number of words written.Predictor variables were classified with respect to their degree of relevance to the measurement construct. The historical content ratings and multiple‐choice history scores were considered to be directly relevant, English composition quality and words written to be indirectly relevant, and neatness and handwriting to be irrelevant. For the two free‐response scores combined, the indirectly relevant variables, in combination, predicted about as well as the directly relevant variables. For the Part A free response, the indirectly relevant variables predicted best, while for the Part B free response, the directly relevant variables predicted best. Handwriting and neatness did not predict free‐response scores well at all.The regression equations developed were also used to estimate over‐ or underprediction by sex of free‐response scores from the AP multiple‐choice score alone and from multiple predictors. As expected, the AP multiple‐choice score underpredicted female performance on the free‐response part of the test and overpredicted male performance. For the AP U.S. History Examination, the over/underprediction effect size was reduced to a nonsignificant level when the English composition quality of the free responses was added to the predictor set. For European History, the over/underprediction effect size was also reduced to a nonsignificant level by the addition of English composition quality, although the reduction of the effect size was not as dramatic as that for U.S. History. No other predictor variables contributed importantly to the reduction of effect sizes.To develop further insight into this test format phenomenon, AP files for the sampled cases were matched with College Board Admissions Testing Program (ATP) data to obtain Scholastic Aptitude Test (SAT) scores and English Composition Test scores. In U.S. History, the SAT‐verbal, Test of Standard Written English (TSWE), and English Composition Test (ECT) scores made significant independent contributions to the prediction of AP free‐response scores over and above that possible using the AP objective (multiple‐choice) score, especially for Part A. In European History, SAT‐verbal scores made significant independent contributions, although TSWE and ECT scores did not.This study suggests that format effects are real and cannot be attributed to bias in scoring or to totally irrelevant variables. When scoring was conducted analytically with a focus on historical content, no sex differences were observed in the free‐response portions. This is the same result observed for the regular administration readings, which are graded holistically and by readers different from those used for this study. The sexes appear to differ in how they respond to two legitimate but different ways of assessing history skills. The study raises the question of how much influence basic skills and aptitude should have on the outcomes of assessments in specific areas of achievement.

AN EXAMINATION OF THE RELATIONSHIPS OF ACADEMIC COURSEWORK WITH ADMISSIONS TEST PERFORMANCE

ABSTRACTThe redesigned Student Descriptive Questionnaire (SDQ) provides a great deal of background information about examinees sitting for the Scholastic Aptitude Test (SAT). One set of questions focuses on the number of years and types of courses in the students' academic background. This information makes it possible to explore the relationships between coursework and performance on the SAT. This study used data from the 1987 National Sample Tape, which contains SDQ responses and score information from 100,000 seniors in the class of 1987. The analyses examined the relationships between both the Admissions Testing Program (ATP) SAT and Achievement Test scores and the type and level of coursework in six academic areas. In order to provide a more accurate representation of these relationships, the data were adjusted to account for differences related to student academic achievement. The results show that coursework in the disciplines of mathematics, natural science, and foreign languages has the strongest adjusted relationships with SAT‐mathematical scores. SAT‐verbal scores appear to be most strongly related to the number of years of foreign language coursework. These relationships are generally consistent across ethnic groups and income levels. However, it appears that the relationships are stronger for students with higher grade‐point averages. The specific courses that seem to be most strongly related to SAT performance are upper‐level courses in mathematics, natural science, and foreign languages. The specific course relationships appear stronger for male than for female examinees. An analysis of Achievement Test data again indicates that upper‐level coursework in the three previously mentioned academic areas is most strongly related to performance on the Achievement Tests. All the research findings must be qualified because the study was exploratory. Student participation in the various courses was based solely on the motivation and the opportunity of the students to take the courses. As a result of this self‐selection, causal interpretations of the results are not appropriate.

Revision of Dental Admission Test standard score scale

In October 1988 the Dental Admission Testing Program will begin reporting DAT scores on a new standard score scale. This scale will be based on the underlying ability metric rather than on the normal distribution that is the basis of the current-1 to 9 scale. A comparison of the two standard score scales is included, as well as a discussion of the advantages offered by the ability based metric.

A STUDY OF THE TEMPORAL STABILITY OF IRT ITEM PARAMETER ESTIMATES1,2

ABSTRACTThis research report focuses on the stability of item response theory (IRT) item parameter estimates when the items are calibrated on two different samples of examinees who have responded to the items at two different points in time, i.e., the temporal stability of the parameter estimates. The data were collected from regular administrations of the College Board Admissions Testing Program Scholastic Aptitude Test (SAT) and Achievement Tests. The three‐parameter logistic model was used to characterize the relationship between the underlying trait and performance on an item and a variety of methods were used to assess parameter estimate stability.Two important conclusions were drawn from the study. One, stability of parameter estimates is clearly related to type of test. Parameter estimates obtained from SAT‐verbal and SAT‐mathematical are more likely to exhibit stability over time than those obtained for achievement tests of Biology or American History and Social Studies. Secondly, lack of stability of item parameter estimates appears to be more closely related to differences in ability among calibration samples than to the lapse of time between administrations of the test, which, in the study, ranged from 14 to 52 months. It should be noted, however, that for the particular tests studied, ability differences in calibration samples appeared to be unrelated to time differences between administrations. This may not be typical of other testing situations, where ability differences in the calibration samples can be directly related to length of time between test administrations. This could be brought about, for example, by changes in curricular emphasis.

LANGUAGE FACTORS AND HISPANIC FRESHMEN'S STUDENT PROFILE

ABSTRACTThis study investigated the role of language factors as potential determinants of Hispanics' educational progress. Hispanic as well as non‐Hispanic educators have argued that if we knew more about Hispanics' language background and language skills, we might improve the assessment of their cognitive and academic skills. More specifically, the purpose of this study was to investigate the relationship of language background factors to measures of college aptitude as measured by the Scholastic Aptitude Test (SAT) and other College Board Admissions Testing Program achievement and placement tests. Language background factors that were studied included amount of exposure to both Spanish and English, frequency of use of the two languages, and proficiency in each language.The students surveyed were freshmen in the fall of 1982 who were attending a sample of four‐year institutions that were participating in the College Board's Summary Reporting Service. The data analyzed included test scores, responses to the Student Descriptive Questionnaire (SDQ), and a special questionnaire supplement that included 36 items dealing with language background and language proficiency questions.The major questions that were addressed by this study were: (1) How did Hispanic freshmen vary with respect to their perceptions of language use and proficiency? (2) What were the relationships between their self‐perceptions of their language skills and their test scores? (3) Were there specific language survey questions that would add to the utility of the SDQ in predicting SAT scores?The principal findings of the study indicated that a large number of non‐SDQ language questions were more highly related to SAT and TSWE test performance than was the existing SDQ question 38 (Is English your best language?). Language survey items probing preferences for using Spanish versus English and freshmen's self‐ratings of English language proficiency seemed to be the most outstanding predictors of SAT‐verbal scores as well as TSWE scores. Many of the language survey questions improved the prediction of the SAT scores by 10 percent or more when used in combination with the SDQ question 38. In addition, the importance of language survey questions as predictors of SAT‐verbal scores varied by Spanish language group, with language survey items being most important for Puerto Ricans.

AN INVESTIGATION OF THE FEASIBILITY OF APPLYING ITEM RESPONSE THEORY TO EQUATE ACHIEVEMENT TESTS1,2,3

ABSTRACTThe purpose of this study was to examine the feasibility of using item response theory (IRT) procedures to equate different forms of three achievement tests (Biology, American History and Social Studies, and Mathematics Level II) that are administered by the College Board Admissions Testing program and one achievement test (Advanced Biology) that is part of the Graduate Record Examinations Achievement Test battery. All of the tests investigated in this study are typically equated using conventional linear or curvilinear (equipercentile) methods.Because the investigators had access to data from large scale testing programs that administer many different forms of a test, it was possible to design the study such that a unique criterion for evaluating the equating results, scale drift, was available. Scale drift will occur if the result of equating, say, Form A to Form C through Form B is not the same as that obtained by equating Form A directly to Form C. This study was designed so that for all the Achievement tests investigated, the drift in each scale when IRT equating was used could be compared to the drift occurring when the conventional linear and equipercentile equating methods, used operationally to put new forms on scale, were applied.The results of the study indicate, using scale drift as the evaluative criterion, that it is feasible to use item response theory to equate the four Achievement tests selected for investigation. The results also indicate that the conventional linear methods typically used to equate the tests perform quite adequately. The question of whether the IRT procedures used in the study are sufficiently robust to violations of the assumption of unidimensionality, or whether Achievement tests, of the type used in this study, give rise to sufficiently unidimensional data, must be resolved before the results of the study can be generalized to other Achievement testing situations.

MULTIVARIATE TALENT FLOW ANALYSIS: A PILOT STUDY1

ABSTRACTThis heuristically‐oriented study employed a wide range of data from the College Board Admissions Testing Program file on college‐bound students to explore a multivariate approach to talent‐flow analysis, or analysis of differences between admissions‐stage groups. Multivariate talent flow analysis, a largely undeveloped area of admissions‐related research, may be thought of as having two related general objectives:a) to analyze the correlates of dichotomous flow criteria—i.e., to determine which of several student variables, or talent descriptors, are associated with filing an application for admission (the flow criterion is application versus nonapplication), being offered admission (accept vs. reject), and enrolling (enrolling vs. not enrolling), respectively; and to assess the relative importance of the student variables as factors affecting, or involved directly or indirectly in, advancement/nonadvancement at each admissions stage; andb) to predict the respective flow criteria—i.e., to determine rates or probabilities of application, acceptance, and enrollment, respectively, according to score‐level on composites of relevant student variables optimally weighted to discriminate between criterion groups at each admissions stage.Results of the exploratory study, using data for college‐bound students sending CB score reports to one State University, strongly suggest that the type of information yielded by using the multiple regression or discriminant model to analyze differences between admissions‐stage groups with respect to a variety of academic, personal, and demographic data (talent descriptors) should be of value to colleges and universities interested in assessing and evaluating their admissions and recruitment policies and procedures. Moreover, the optimally weighted discriminant composite of variables found to differentiate admissions‐stage groups provides information that can be used to facilitate prediction of yield—e.g., to determine which candidates are most likely to complete an application or to enroll if offered admission.From the point of view of accountability in admissions, systematic analysis of the correlates of the accept/reject decision would appear to be relevant. By regressing the accept/reject criterion on appropriate student variables it would be possible to ascertain the relative role in admissions of, say, academic variables (such as tested ability and previous academic record) and nonacademic and/or socially relevant variables (such as, for example, socioeconomic status, sex, minority vs. nonminority status).Knowledge of the correlates of individual decisions to apply or to enroll if accepted should be useful to institutions interested in identifying high‐yield candidate categories in connection with their recruitment programs.Analysis of student data from a variety of colleges and universities representing different admissions modes and circumstances is needed to establish the extent and nature of variation in the patterns and the correlates of application, admission, and enrollment according to institutional differences in admissions policies and emphases.

An Overview of the Mathematics Achievement Tests Offered in the Admissions Testing Program of the College Entrance Examination Board

In the 1964-65 academic year, the College Entrance Examination Board introduced two new levels of mathematics Achievement Tests in its Admissions Testing Program (ATP): Mathematics Level I and Mathematics Level II. They replaced the intermediate and advanced mathematics examinations in response to factors affecting mathematics education from the mid-1950s to the mid-1960s.

Comparison of males and females in the dental admission testing program

Comparison of males and females in the dental admission testing program

Substitution of perceptual‐motor ability test for chalk carving in Dental Admission Testing Program

Substitution of perceptual‐motor ability test for chalk carving in Dental Admission Testing Program

THE EFFECTIVENESS OF AN ESSAY SECTION IN THE AMERICAN HISTORY AND SOCIAL STUDIES TEST1

ABSTRACTThe contribution of a 20‐minute essay question, given as part of the one‐hour achievement test in American History and Social Studies, to the pool of information available on a candidate from an all‐objective examination of the College Board Admissions Testing Program, is presented in this report. The study limits itself to a consideration of the psychometric issues only. It does not deal with other important issues related to the educational impact of the essay or to the implications of the cost of introducing and maintaining a reliable essay section in the American History and Social Studies test. The results of the study led to the conclusion that the additional information gained by the introduction of an essay section is minimal. Moreover, the correlation of the essay with various subtests of the American History and Social Studies objective test suggests that the obtained essay scores may have reflected more of the factual content of the answers than the other skills.

The Dental Admission Testing Program

The Dental Admission Testing Program

The Dental Admission Testing Program and proposed changes

Journal of Dental EducationVolume 35, Issue 4 p. 237-242 Article The Dental Admission Testing Program and proposed changes RA Pyskacek, RA PyskacekSearch for more papers by this author RA Pyskacek, RA PyskacekSearch for more papers by this author First published: 01 April 1971 https://doi.org/10.1002/j.0022-0337.1971.35.4.tb00354.xCitations: 2About ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat No abstract is available for this article.Citing Literature Volume35, Issue4April 1971Pages 237-242 RelatedInformation