An investigation of the variability of judgments of experienced and inexperienced listeners in their use of a screening test of articulation

Abstract

This study examines the extent to which the mean, or average degree of agreement, of a group of listeners actually represents the consistency of judgments of all members of that group. Twenty-two experienced and inexperienced judges scored the responses, from tape recordings, of six children with multiple articulation errors on the McDonald Screening Deep Test of Articulation. The intra- and interexaminer reliability of the judges was assessed. Results suggested that, as a group, both experienced and inexperienced listeners were capable of making consistent decisions about children's responses on the Screening Deep Test of Articulation. It was determined, however, that group averages may not reflect accurately the consistency of all members of the group or accurately estimate the population average. Variances for both groups were high and significant differences were found between the variances of the groups. Confidence intervals were reported and their clinical implications discussed.