Screening Test Items Research Articles

Assessing the Impact of a Test Question: Evidence from the “Underground Railroad” Controversy

AbstractOn the second day of a 2019 high‐stakes English Language Arts assessment, Massachusetts 10th graders faced an essay question that was based on a passage from the novel “The Underground Railroad” and publicly characterized as racially insensitive. Though the state excluded the essay responses from student scores, an unresolved public controversy focused on whether this question created a racial bias in performance on the remaining test items. We present the results from an independent, preregistered study of this question. Our confirmatory results indicate that exposure to the controversial question is associated with a small reduction in the comparative performance of Black students on the overall test (approximately .006σ). However, we also find a wide dispersion of such effects when examining similarly small sets of test items from prior state assessments that lacked a controversial question, which suggests the 2019 assessment was not distinctive. Our approach offers a potential template that may be useful in other contexts where testing controversies occur and underscores the importance of carefully screening test items to avoid such occurrences.

Validation of the fine motor subtest of the Bayley-III with children with sickle cell disease using Rasch analysis.

Children with sickle cell disease (SCD) are at risk for fine motor (FM) delays; however, screening for FM impairments is not common among young children with SCD. The Bayley Scales of Infant and Toddler Development, Third Edition (Bayley-III) is the most commonly used performance-based developmental assessment. We aim to determine if the FM subtest of the Bayley-III is structured hierarchically in accordance with development and comprehensively evaluates FM development in children with SCD. Bayley-III assessments were completed between October 2009 and December 2013. The Bayley-III FM screening test, a shorter and more rapid method of assessing for FM impairments, was not directly administered to participants. Screening test scores were calculated from full Bayley-III scores. Rasch analysis was performed using WINSTEPS. Sixty children with SCD were included in the final Rasch model. The Rasch-generated Wright map, which jointly positions items and persons on the same latent trait, illustrated that the FM items were slightly skewed towards more challenging items, indicating more difficult items may be overrepresented. High item separation values were reported (17.4), and item outfit statistics were less than 1.7. More than one third of items demonstrated overfit, indicating possible item redundancy. The FM subtest and the screening test, a shorter and faster method of assessing skills, were highly correlated (r = 0.993, p < 0.001). The Bayley-III FM subtest is structured hierarchically, aligning with motor development, and comprehensively evaluates FM development in children with SCD. The test could be improved by reordering items, removing overfitting items and modifying screening test items to capture all ranges of development. The screening test is comprehensive and has high potential clinical utility among children with SCD.

Developing an Arabic Screening Test for Adult-Onset Chronic Aphasia

Aim: To develop a time-efficient Arabic test battery for adult-onset chronic aphasia that provides information about the type and the severity of the disorder. Patient and Methods: A total of 90 participants were recruited and divided into 3 groups. The groups consisted of 30 adult-onset post-stroke chronic aphasic patients, 30 adult non-aphasic adults with central neurological disorders, and a group of 30 non-brain-damaged healthy adults. All participants were assessed using the Mansoura Arabic Screening Aphasia Test (MASAT), which consists of 4 main sections: (1) language expression abilities including repetition and naming items, (2) language comprehension questions, (3) fluency, and (4) reading, writing and calculation items. The content validity, internal consistency, clinical validity, and convergent validity of the MASAT were evaluated. Results: The MASAT demonstrated statistically high reliability and validity. The high α-values in all subtotal scores among the 3 groups were judged to denote excellent intercorrelation among the screening test items. Conclusion: The MASAT is a valid and a reliable brief assessment tool that can be completed on the first clinic appointment that detects the type and severity of Arabic-speaking aphasic patients.

노인 경도 인지장애를 위한 선별평가 항목 개발

노인 경도 인지장애를 위한 선별평가 항목 개발

Development of Test Toolkit of Hard Review to Evaluate a Random Clinical Decision Support System for the Management of Chronic Adult Diseases

Over 30 % of Korean population suffers from chronic diseases, and the management cost of this portion of the population is estimated to be more than 7 % of the gross domestic product and is expected to continue to increase with the aging of society. The u-healthcare (ubiquitous healthcare)-based home healthcare service has been emerging as a solution to this issue. Clinical decision support system (CDSS) is software that provides professional medical knowledge to help people make medical decisions, and is required for the efficient operation of u-healthcare as a core technology. U-healthcare-based CDSS is meant to improve accessibility of medical information for the public, but it at the same time carries a substantial risk of misuse. Hence, a certain level of standards and specifications is required to guarantee the quality of u-healthcare-based CDSS to protect public health. In this context, this project aimed to develop technologies to evaluate reliabilities of u-healthcare-based CDSS dealing with diabetes mellitus and hypertension. This study created a Test Toolkit to evaluate random CDSS with a function to confirm results obtained from CDSS performance evaluation technologies. Results of CDSS were obtained based on screening test items. The design descriptions are defined as (1) creation of test scenario to test CDSS, (2) grading function using returned answers to CDSS, and (3) confirmation of test result. The functions of the Test Toolkit are divided into two types. (1) Test scenarios and the answers to the scenarios are created by reading and fabricating standard scenarios from the DB based on test items input by the evaluator. (2) Appraisal CDSS creates a file of result values using a given test scenario file, and then a CDSS evaluation report, classified by test item, is formed by comparing answers to those found in (1). It should be noted that functions (1) and (2) are completely independent. Conclusively, researchers developed a toolkit to evaluate clinical reliabilities and software stabilities of CDSS for healthcare free from misjudgments caused by human factors and to speed up overall evaluation processes tremendously and efficiently. This toolkit will ensure proper usage of CDSS by the public for health benefits.

Screening Test Items for Differential Item Functioning

A method for medical screening is adapted to differential item functioning (DIF). Its essential elements are explicit declarations of the level of DIF that is acceptable and of the loss function that quantifies the consequences of the two kinds of inappropriate classification of an item. Instead of a single level and a single function, sets of plausible levels and plausible functions may be declared. Some extensions are outlined, and a simulation study is presented.

The Optimal Sequence and Selection of Screening Test Items to Predict Fall Risk in Older Disabled Women: The Women's Health and Aging Study

Falls are a major cause of disability, dependence, and death in older people. Brief screening algorithms may be helpful in identifying risk and leading to more detailed assessment. Our aim was to determine the most effective sequence of falls screening test items from a wide selection of recommended items including self-report and performance tests, and to compare performance with other published guidelines. Data were from a prospective, age-stratified, cohort study. Participants were 1002 community-dwelling women aged 65 years old or older, experiencing at least some mild disability. Assessments of fall risk factors were conducted in participants' homes. Fall outcomes were collected at 6 monthly intervals. Algorithms were built for prediction of any fall over a 12-month period using tree classification with cross-set validation. Algorithms using performance tests provided the best prediction of fall events, and achieved moderate to strong performance when compared to commonly accepted benchmarks. The items selected by the best performing algorithm were the number of falls in the last year and, in selected subpopulations, frequency of difficulty balancing while walking, a 4 m walking speed test, body mass index, and a test of knee extensor strength. The algorithm performed better than that from the American Geriatric Society/British Geriatric Society/American Academy of Orthopaedic Surgeons and other guidance, although these findings should be treated with caution. Suggestions are made on the type, number, and sequence of tests that could be used to maximize estimation of the probability of falling in older disabled women.

Development of the Miller Screening for Preschoolers

This paper describes the development and initial validation of the Miller Screening for Preschoolers (MSP). A pilot edition of the test was administered to 174 preschool children in the state of Colorado. Of these, 39 had been previously identified as developmentally delayed. On the basis of preliminary analyses, test items were discarded if they were insensitive to differences between the at-risk group and the not-at-risk group or if they were nonsignificantly related to developmental age trends; if they were only weakly related to the domain total score; if they were redundant with other items within a domain; if they were too easy or too difficult for any given age; if they were rated low by the testers on administration and scoring issues; or if the test materials were deemed to be too expensive. This paper presents final statistics for the selected screening test items and discusses implications for the design, construction, and validation of developmental tests.

Validation of key Denver developmental screening test items: A preliminary study: Frankenburg, W., Ker, C., Engelke, S., Schaefer, E., & Thornton, S. (1988). The Journal of Pediatrics, 112, 560–566

Validation of key Denver developmental screening test items: A preliminary study: Frankenburg, W., Ker, C., Engelke, S., Schaefer, E., & Thornton, S. (1988). The Journal of Pediatrics, 112, 560–566

Validation of key Denver Developmental Screening Test items: A preliminary study

Three sequential studies were undertaken to identify a subset of 39 key Denver Developmental Screening Test (DDST) items covering the ages from birth to 6 years. The first study used binary integer programming with a sample of 2343 children evaluated with the DDST in Denver's Neighborhood Health Program. It identified 35 DDST items that would identify 100% abnormal and 97.9% abnormal and questionable DDSTs. A further validation with 113 Pitt County, North Carolina, children indicated that the addition of four more DDST items to the previous 35 would identify 100% abnormal and 92.6% abnormal and questionable DDSTs. A third study cross-validating the 39 items with a sample of 180 children at "high risk" identified 100% abnormal and 91.7% abnormal and questionable DDSTs. The 39 key DDST items ranged throughout the entire age span and among the DDST's four sectors. Use requires the administration of approximately four items at any one age, on average. Those children with suspect scores on the key DDST items (about 19% of a low-income population) should be screened with the remainder of the DDST to decrease overreferrals. For accuracy, the key DDST items must be administered in the manner prescribed in the DDST manual.

振動障害の第一次健診項目の信頼性と妥当性の評価

Six primary screening test items (grip strength, skin temperature, Nail-Press test, pain sense, and vibration sense at 125 Hz and at 250 Hz) for the vibration syndrome were assessed for validity and reliability using four measures: correlation with working time in years with vibrating tools; degree of relationship with four major complaints (Reynaud's phenomenon, numbness, coldness, and pain); correlation between two repeated examinations; values of t-test calculated on two groups of workers of comparable age, one having all of the four major complaints and the other having no complaint at all. Both left and right hand scores were used in the analysis with a total of 12 items. As these scores were too heterogeneous to make direct comparison, the screening items were ranked according to the values of the four measures and then the obtained ranks were totalled for each item to establish a further ranking. In this way, the ranks of each screening item were determined.The results indicated that: 1) For validity, pain sense and 250 Hz vibration sense were found to have relatively high validity, followed by 125 Hz vibration sense and grip force. Skin temperature and the Nail-Press test occupied the lowest ranks. 2) For reliability, grip force and 250 Hz vibration sense were highest followed by pain sense. In this case again, skin temperature and the Nail-Press test turned out to belong to the lowest group, which included the 125 Hz vibration sense. 3) The right and left hand scores were separately analyzed and the right hand scores tended to surpass the left hand ones, although these comparison were not statistically significant (in 8 out of 12 comparisons, right hand scores were superior to left hand ones for validity, and for reliability it was the case for 9 out of 12 cases).These results suggest that the two measures of the peripheral blood flow (i.e., skin temperature and the Nail-Press test) should be replaced by more reliable measures.

Characteristics of Child Development in Okinawa; The Comparisons with Tokyo and Denver and the Implications for the Developmental, Screening.

Developmental differences in the Denver Developmental Screening Test items were demonstrated between the 615 Okinawa( Miyako and Yaeyama Islands) chi 1 dren and the 1171 Tokyo children. The children in Tokyo were generally advanced than those in Okinawa after one year of age, while the children in Okinawa demonstrated earlier development in the middle and late periods of the first year. There were also interesting differences between Okinawa and Denver. Okinawa children were initially slow in gross motor development. Denver children were markedly advanced in some of the language items. Ecological and socio-cultural factors which may contribute to these differences as well as the implications of these results for the developmental screening were discussed.