Autism Knowledge Assessments: A Closer Examination of Validity by Autism Experts.

Abstract

The goal of the current study was to conduct a substantive validity review of four autism knowledge assessments with prior psychometric support (Gillespie-Lynch in JAutism and Dev Disord 45(8):2553-2566,2015; Harrison in J Autism and Dev Disord 47(10):3281-3295, 2017; McClain inJ Autism and Dev Disord 50(3):998-1006, 2020; McMahon inRes Autism Spectr Disord 71:101499, 2020).69 autism experts who served on the editorial board of one or more peer-reviewed autism journals evaluated the accuracy and ambiguity of autism knowledge questions.34% of the questions were flagged as "potentially problematic" for accuracy, and 17% of the questions were flagged as "potentially problematic" for ambiguity. Autism expert feedback revealed three themes across ambiguous questions: (1) an oversimplification of mixed or still-evolving research literature, (2) an insufficient recognition of the heterogeneity of the autism spectrum, and (3) a lack of clarity in the question/answer prompt.Substantive validity of future autism knowledge assessments should be carefully evaluated via feedback from a diverse group of autism experts and/or potential respondents. Potentially problematic questions can be removed or modified to improve the validity of autism knowledge assessments.