Conditions Of Test Administration Research Articles

Reliability and Feasibility of Administering a Child Language Assessment via Telehealth.

Despite the speed with which telehealth use advanced during the COVID-19 pandemic, evidence is needed to support the remote delivery of standardized assessments. This study investigated the reliability and feasibility of administering a standardized language assessment administered in real-world telehealth scenarios compared to in-person administration. A total of 100 children between the ages of 3 and 12 years were administered one of three versions of the Clinical Evaluation of Language Fundamentals (CELF). Children were administered the CELF by the same licensed speech-language pathologists (SLPs) in person and using telehealth, with the order counterbalanced. Means for Core Language standard scores were compared between conditions and among devices. Descriptive statistics summarized the behavior and technology disruptions during administration as well as the results of parent and SLP telehealth perception surveys. In-person and telehealth mean scores on all three versions of the CELF revealed no systematic differences of one condition under- or overestimating another. The incidence of child behavior disruptions was similar for both test administration conditions. Adaptations compensated for the rare technology disruptions. Despite no significant language score and behavior differences between testing conditions, parents reported they continued to prefer in-person assessments. SLP participants viewed telehealth overall positively but identified conditions in which they continued to prefer in-person delivery. This study provides evidence of minimal or no differences in scores and behavioral or technological disruptions between remote and in-person administration of the CELF core language assessments. SLP and parent participants' attitudes toward remote delivery of standardized tests appear to be evolving in a positive direction compared to previous studies. https://doi.org/10.23641/asha.25292752.

The Use of Telepractice to Administer Norm-Referenced Communication and Cognition Assessments in Children With Hearing Loss: A Rapid Review.

Telepractice administration of norm-referenced assessments of communication and cognition is relatively new, and evidence to support this practice for children with hearing loss is limited. This rapid review examines the validity, reliability, feasibility, and common features of telepractice-administered norm-referenced assessments of communication and cognition for children with hearing loss to determine whether results via telepractice-administration are usable. This rapid review was conducted in accordance with the Cochrane Collaboration Rapid Reviews Methods Group recommendations. Rayyan software was used for initial and full-text screening, and the Quality Assessment of Diagnostic Accuracy Studies was used to measure study quality. Electronic databases searches identified two studies that met the eligibility criteria. The findings of this rapid review provide some evidence to indicate that results of norm-referenced assessments do not differ depending on test administration conditions or hearing group status. Although both studies were of sufficient quality, replicability of these studies is uncertain due to the limited description of telepractice administration procedures. Participants indicated that sound levels provided by inbuilt computer speakers were adequate for them to be able to participate in telepractice assessment. Telepractice administration of the norm-referenced assessments in the included studies may be valid, reliable, and feasible; however, the generalizability of these findings to other norm-referenced assessments is uncertain due to the limited amount of research.

Mode Effects in College Admissions Testing and Differential Speededness as a Possible Explanation

AbstractAs testing programs transition from paper to online testing, they must study mode comparability to support the exchangeability of scores from different testing modes. To that end, a series of three mode comparability studies was conducted during the 2019–2020 academic year with examinees randomly assigned to take the ACT college admissions exam on paper or online. Unlike many prior mode comparability studies, examinees testing online earned higher scores on average than examinees testing on paper, especially on the English, reading, and writing tests. Much of the observed mode effects were accounted for by items near the ends of the tests. Subsequent examination of item omit rates, differential item functioning by mode, and change‐point analyses supported the notion that the observed mode effects were related to differential speededness. Specifically, examinees testing online were slightly less speeded, and were therefore able to perform better on items near the ends of the tests. Practical implications for linking across modes, test design, and test administration conditions are discussed.

Test Administration Methods and Cognitive Test Scores in Older Adults with Hearing Loss

In light of the high prevalence of hearing loss and cognitive impairment in the aging population, it is important to know how cognitive tests should be administered for older adults with hearing loss. The purpose of the present study is to examine this question with a cognitive screening test and a working memory test. Specifically, we asked the following questions in 2 experiments. First, does a controlled amplification method affect cognitive test scores? Second, does test modality (visual vs. auditory) impact cognitive test scores? Three test administration conditions were created for both Montreal Cognitive Assessment (MoCA) and working memory test (a word recognition and recall test): auditory amplified, auditory unamplified, and visual. The auditory administration was implemented through a computer program to control for presentation level while the visual condition was implemented through timed computer slides. Data were collected from older individuals with mild-to-moderate sensorineural hearing loss. We did not find any effect of amplification or test modality on the total score of the cognitive screening test (i.e., MoCA). Amplification improved working memory performance as measured by word recall performance, but test modality (auditory vs. visual) did not. These results are consistent with literature in demonstrating a downstream effect of audibility on working memory performance. From a clinical perspective, these findings are informative for developing clinical administration protocols of these tests for older individuals with hearing loss.

A Meta-Analysis on Gender Differences in Mental Rotation Ability Measured by the Purdue Spatial Visualization Tests: Visualization of Rotations (PSVT:R)

This meta-analysis was conducted to estimate the magnitude of gender difference in three-dimensional (3-D) mental rotation ability and to investigate how factors related to test administration conditions play a role in varying gender difference effect sizes and threatening validity. Individuals’ 3-D mental rotation ability was measured by the Purdue Spatial Visualization Tests: Visualization of Rotations (PSVT:R). We integrated 70 effect sizes of gender differences in mental rotation ability measured by the PSVT:R which were obtained from 40 primary studies. The results indicated that male participants outperformed females on the test (Hedges’ g = 0.57). The I 2 statistic indicated 41.7 % of variation in effect sizes reflects real heterogeneity. The moderator analysis indicated that male superiority on spatial ability tasks measured by the PSVT:R is related to the implementation of time limits. The gender difference became larger when stringent time limits (equal or less than 30 s per item) were implemented.

A Comparison of Computer-Based Testing and Pencil-and-Paper Testing for Students with a Read-Aloud Accommodation

This study examined the academic performance and preference of students with disabilities for two types of test administration conditions, computer-based testing (CBT) and pencil-and-paper testing (PPT). Data from a large-scale assessment program were used to examine differences between CBT and PPT academic performance for third to eleventh grade students with a read-aloud accommodation in reading, mathematics, and science. Since random assignment was not possible, propensity score analyses were used to establish equivalent groups and to test for differences in performance. Students in the PPT condition with an adult reader had higher mean scores in almost all academic content areas than those with the CBT read-aloud condition, with effect sizes ranging from extremely small ( d = .02) to moderate ( d = .69). Differential item functioning (DIF) analyses suggested that most items had negligible DIF and did not favor either the CBT or the PPT conditions. Students and staff reported that students preferred the CBT to the PPT, and students believed they performed better using the computer. Additional research that controls extraneous factors such as instructional time and familiarity with testing environment is recommended to better evaluate the relationship between testing modes and academic performance.

Impairments of Colour Vision Induced by Organic Solvents: A Meta-Analysis Study

The impairment of colour discrimination induced by occupational exposure to toluene, styrene and mixtures of organic solvents is reviewed and analysed using a meta-analytical approach. Thirty-nine studies were surveyed covering a wide range of exposure conditions. Those studies using the Lanthony Panel D-15 desaturated test (D-15d) were further considered. From these for 15 samples data on colour discrimination ability (Colour Confusion Index, CCI) and exposure levels were provided, required for the meta-analysis. In accordance with previously reported higher CCI values for the exposed groups, the computations yielded positive effect sizes for 13 of the 15 samples, indicating that in the great majority of the studies the exposed groups showed inferior colour discrimination. However, the meta-analysis showed great variation in effect sizes across the studies. Possible reasons for inconsistency among the reported findings are discussed. These pertain to exposure-related parameters, as well as to confounders such as conditions of test administration and characteristics of subject samples. Those factors vary considerably among the studies and might have greatly contributed to divergence in measured colour vision capacity, thereby obscuring consistent effects of organic solvents on colour discrimination.

The influence of shyness on children's test performance.

Research has shown that shy children differ from their peers not only in their use of language in routine social encounters but also in formal assessments of their language development, including psychometric tests of vocabulary. There has been little examination of factors contributing to these individual differences. To investigate cognitive-competence and social anxiety interpretations of differences in children's performance on tests of vocabulary. To examine the performance of shy and less shy children under different conditions of test administration, individually with an examiner or among their peers within the familiar classroom setting. The sample consisted of 240 Year 5 pupils (122 male, 118 female) from 24 primary schools. Shy and less shy children, identified by teacher nomination and checklist ratings, completed vocabulary and mental arithmetic tests in one of three conditions, in a between-subjects design. The conditions varied individual and group administration, and oral and written responses. The conditions of test administration influenced the vocabulary test performance of shy children. They performed significantly more poorly than their peers in the two face-to-face conditions but not in the group test condition. A comparable trend for the arithmetic test was not statistically significant. Across the sample as a whole, shyness correlated significantly with test scores. Shyness does influence children's cognitive test performance and its impact is larger when children are tested face-to-face rather than in a more anonymous group setting. The results are of significance for theories of shyness and have implications for the assessment of schoolchildren.

Wisconsin Card Sorting Test performance in patients with focal frontal and posterior brain damage: effects of lesion location and test structure on separable cognitive processes

Forty-six patients with single focal lesions (35 frontal, 11 nonfrontal) were administered the Wisconsin Card Sorting Test (WCST) under three conditions of test administration. The three conditions varied in the amount of external support provided via specificity of instructions. The WCST, while a multifactorial test, is specifically sensitive to the effects of frontal lobe damage if deficits in language comprehension and visual–spatial search are controlled. There is also specificity of functioning within the frontal lobes: patients with inferior medial frontal lesions, unilateral or bilateral, were not impaired on the standard measures although they had increased loss of set when informed of the sorting categories. Verbal instructions may provide a probe to improve diagnosis and prognosis, assessment of the potential efficacy of treatment, and the time frame of plasticity of specific cognitive operations.

Accommodating Students with Disabilities on Large-Scale Tests: An Experimental Study

In this study, fourth-grade special and general education students took a large-scale state-wide test using standard test administration procedures and two major accommodations addressing response conditions and test administration. On both reading and math tests, students bubbled in answers on a separate sheet (the standard condition) for half the test and marked the test booklet directly (the accommodated condition) for the other half of the test. For a subgroup of students, the math test was read to them by a trained teacher. Although no differences were found in the response conditions, an interaction was found in the test administration conditions (orally reading the test), supporting this accommodation for students with disabilities.

Raising the Stakes of Test Administration: The Impact on Student Performance on the National Assessment of Educational Progress

This article briefly reviews the current discussion of the effects of test administration conditions (i.e., testing stakes), and the motivational levels associated with them, on achievement test performance. The non-experimental study presented here investigates whether differences in test administration conditions and presumed levels of motivation engendered by different testing environments affect student performance on National Assessment of Educational Progress (NAEP) administrations. The testing conditions under study are the "low-stakes" environment of the current NAG administration and a higher stakes environment typified by many state assessment programs. The results suggest that in comparison to a "moderate-stakes" testing environment NAEP does not seriously underestimate achievement levels. However, the results cannot lead to the conclusion that student achievement is unrelated to testing stakes. Nor can one conclude that substantially raising the stakes of NAEP would not be accompanied by an increase in achievement scores.

An Empirical Study of Computerized Adaptive Test Administration Conditions

This empirical study was designed to determine the impact of computerized adap‐ tive test (CAT) administration formats on student performance. Students in medical technology programs took a paper‐and‐pencil and an individualized, computerized adaptive test. Students were randomly assigned to adaptive test administration for‐ mats to ascertain the effect on student performance of altering: (a) the difficulty of the first item, (b) the targeted level of test difficulty, (c) minimum test length, and (d) the opportunity to control the test. Computerized adaptive test data were analyzed with ANCO VA. The paper‐and.pencil test was used as a covariate to equalize abil‐ ity variance among cells. The only significant main effect was for opportunity to control the test. There were no significant interactions among test administration formats. This study provides evidence concerning adjusting traditional computer‐ ized adaptive testing to more familiar testing modalities.

Criterion validity of speech intelligibility rating-scale procedures for the hearing-impaired population.

The criterion validity and reliability of two popular rating-scale procedures for the assessment of the contextual speech intelligibility of hearing-impaired individuals (The NTID Scales) were studied under clinically typical conditions of test administration and evaluation. The criterion measure was a write-down procedure based on subjects' readings of sentence lists chosen from The CID Everyday Sentence lists. Although the results revealed generally high overall validity and reliability coefficients for the two scales, a close examination of the distribution of estimation error over the full range of rating-scale values revealed gross violations of measurement prediction within the clinically most frequent midrange of speech intelligibility. The results indicate that the rating-scale procedure significantly compromises clinical and research classification of an individual's speech intelligibility and that write-down procedures may provide a viable and significantly more accurate alternative for speech intelligibility assessment.

Instructional Set and Thematic Apperception Test Validity

The strong controversy over the Thematic Apperception Test's (TAT) validity may be partly due to the divergent results critics and advocates have obtained in their own research. It is noted that conditions of test administration are closely associated with quality of TAT research results. To demonstrate a cause of TAT invalidity, 199 adolescents were given the TAT following one of four instructional sets: neutral, following a personality test, emphasizing that the TAT is a personality test, and in a nonthreatening but structured setting. As expected, stories written after neutral instructions were valid predictors of need for achievement, affiliation, and power criteria; whereas other stories yielded nonsignificant and sometimes negative validities. It was concluded that the instructions are crucial to the quality of TAT results, and suggestions were offered to help ensure validity.

Responses of normal and learning disabled children as a function of the stopwatch in the matching familiar figures testing situation

To ascertain whether normal and hyperactive learning disabled children differ in their responses to Kagan's Matching Familiar Figures Test (MFF) as a function of a temporal cue, the stopwatch used in the testing situation, 16 learning disabled and 15 normal 8- and 9-year-old boys were compared. A counterbalanced design, in which each child was administered one-half of the test with a stopwatch (standard administration procedure) and the other one-half of the test without a stopwatch (No-Stopwatch procedure), was used. For the normal children, the MFF latency scores in the two conditions were not correlated, indicating that the stopwatch influenced their performance. For the learning disabled children, however, the MFF latency scores in the two conditions were correlated. A significant difference between the two correlation coefficients indicated that the normal and learning disabled children responded in a different manner to the temporal cue. On the MFF error measure, the correlation between the scores obtained in the two conditions of test administration was significant for the normal children, but not for the learning disabled children. A significant difference between these correlation coefficients again indicated differential responding to the temporal cue by the two groups.

Language Dialect, Age, and Intelligence-Test Performance in Disadvantaged Black Children

QUAY, LORENE C. Language Dialect, Age, and Intelligence-Test Performance in Disadvantaged Black Children. CmLD DEVELOPMENT, 1974, 45, 463-468. The effects of administering the Binet in Negro nonstandard dialect and standard English were compared. Subjects were 104 thirdand sixth-grade disadvantaged black children. 52 children from each grade were randomly placed by sex into 1 of 2 conditions of test administration: nonstandard dialect or standard English. A 2 (dialect condition) X 2 (sex of S) x 2 (age level) analysis of variance of IQ scores yielded no significant effect of dialect condition or sex. There was a significant age effect, with third-grade children making higher IQ scores than sixth-grade children. There were no significant interactions. Comparisons of item difficulty yielded no consistent differences between nonstandard-dialect and standard-English test administrations.

The effect of social desirability on two dimensions of self-evaluation in two cultures.

Summary The effect of change in conditions of test administration on self-reported introversion-extraversion and neuroticism was investigated at a North American, Central American, and South American university. Standard conditions, based on standard test norms, were compared with a condition in which S knew that self-report would be compared with reports about him by three well known Ss named by him. Self-reported scores and others' reports agreed with standard test norms on introversion-extraversion, but self-reported neuroticism was significantly higher compared to test norms and others' reports. There were no significant differences between universities. The differential effect on self-reported neuroticism was interpreted in terms of social desirability response set.

An MMPI Scale to Measure Scholastic Personality in Women

On the basis of response frequencies, 68 MMPI items significantly differentiated in each of two analyses between 60 National Merit Semi-finalists and 60 college freshman women. Inspection of the 68 items suggests two general types of items—those showing an emphasis on intellectual pursuit and those involving a denial of any worries or difficulties. Maturational differences and different conditions of test administration may have produced some of the differences in response frequencies between criterion and comparison Ss. Nevertheless, there is a definite indication that high scholastic potential of women can be predicted from a scale consisting of personality inventory items.

EFFECTS OF REINFORCEMENT ON STANDARDIZED TEST PERFORMANCE1

The effects of two different motivational conditions upon standardized test performance were explored for two student populations. The first study involving 12 trainable retardates showed a significant increase in score on the Metropolitan Readiness Test given under reinforcement conditions when these results were compared with scores taken under standard testing conditions. In a second study, these same results were obtained with a group of 30 normal fourth-graders. An additional study was conducted to determine the effect of different experiences with token reinforcement procedures on test performance. It was found that a group of children with six weeks' exposure to reinforcement for daily academic performance scored higher under both conditions of test administration (standard and reinforcement) than a control group. However, in a single exposure to token reinforcement for correct performance on the Metropolitan Test, both the experimental group and its match control showed a parallel increase in test performance. These findings offer a procedure that yields a more representative assessment of a student's academic achievement than does testing under standard conditions.

Evaluational set and creativity.

To test Wallach and Kogan's hypothesis that low creativity is associated with fear of evaluation, a set of creativity tests with instructions meant to induce low, high, and neutral evaluational sets was administered to 147 fifth and sixth graders. No effect of evaluational set was found. Girls performed significantly better than boys on all measures of creativity. Reading skill was strongly correlated with creativity scores for our conditions of test administration. Subtests were significantly intercorrelated.