Beware of Intelligence Results Based on Common Verbal Tests

Abstract

Intelligence (IQ) tests are used to determine a child's intellectual abilities relative to his or her age group. Results are used to diagnose learning disabilities, intellectual disabilities, and giftedness; and to inform psychoeducational interventions and student education plans. The majority of tasks on intelligence tests require vision, such as visual pattern recognition. Therefore, psychologists typically use verbal sections, especially vocabulary tests, to assess students who have visual impairments. In a recent study (Morash & McKerracher, 2016), we investigated whether a common vocabulary test produced scores from children with visual impairments that were reliable. We found that reliability was not high enough to be used for high-stakes decisions, such as determining diagnoses or educational programming. Therefore, we do not recommend common verbal tests developed for sighted children be used with children who have visual impairments. Practitioners need to be wary and watchful of any diagnoses based on inappropriate intelligence measures. Previously noted issues with sighted-normed verbal tests Prior to our study, other researchers had not considered whether administering common verbal tests to children with visual impairments may produce scores with low reliability (see, for example, Crepeau-Hobson & Vujeva, 2012; Nelson, Dial, & Joyce, 2002). Instead the literature noted that using only verbal tests neglects important nonverbal abilities, such as spatial reasoning, which results in an incomplete assessment of intellectual functioning (Bauman & Kropf, 1979; Dekker, 1993; Gutterman, Ward, & Genshaft, 1985; Lund, Miller, & Ganz, 2014; Miller & Skillman, 2003). In addition, scores on common verbal tests are compared to normative data (norms) from sighted children, which may not be appropriate (see, for example, Dekker, 1993; Hull & Mason, 1995; Tillman & Bashaw, 1968; Vander Kolk, 1977). Comparison to norms reveals whether the child scored below or above average for his age group, and by how much; resulting in an IQ score and possible diagnosis of learning disability, intellectual disability, or giftedness. However, children with visual impairments tend to score higher or lower than their sighted peers on many tasks, for reasons other than their intelligence. For example, they tend to score higher on tests of verbal working memory (see, for example, Hull & Mason, 1995), presumably due to daily practice using verbal working memory strategies, such as counting steps, to complete tasks without vision. Although scores may be inflated or deflated due to using sighted norms, it has been suggested that sighted-normed verbal tests can be useful in providing insight into the child's functioning relative to sighted peers (Dial & Dial, 2010), to track a child's performance over time, and to compare children with visual impairments to one another. Unfortunately, these uses are invalidated by low reliability. Low reliability of sighted-normed verbal tests Reliability refers to the consistency of test scores. Low reliability is a separate, and possibly more severe, problem than inflated or deflated scores. If test scores have low reliability, an unacceptably large part of the score is due to random error--as if a coin flip determines a large part of a child's score and diagnosis. Scores with low reliability have limited value for comparing performance over time and across children, since psychologists cannot trust that scores reflect meaningful variations instead of random errors. Although common verbal tests produce scores with good reliability for sighted children, they are not guaranteed to do so with children who have visual impairments (Loevinger, 1957). It is not unreasonable to expect differential functioning, because children who have visual impairments can experience atypical conceptual and semantic development (see, for example, Pring, Freestone, & Katan, 1988). …