PRO129 IMPORTANCE OF UNCERTAINTY IN MEAN SCORE ESTIMATES OF ONE SURVEY QUESTION ABOUT TEST ACCURACY: AMERICAN COLLEGE OF MEDICAL GENETICS (ACMG) RECOMMENDATIONS FOR NEWBORN SCREENING (NBS) IN MEDIUM CHAIN KETOACYL-COA THIOLASE DEFICIENCY (MCKAT)

Abstract

In 2006 ACMG surveyed stakeholders to make recommendations for 84 NBS conditions. A controlling question was (SS): “was a sensitive and specific test for the condition available?” Scoring was 0/200 for No/Yes. With a mean less than 100, the condition was not recommended. We sought to assess the importance of uncertainty around the SS score (score reported, “65% of the maximum” (200)). We examined uncertainty with missing data and sampling variability, both ignored by ACMG. As missing responses (MR) were not reported for SS, we tested possible score proportions consistent with 65% to determine potential MR and score boundary estimates without making assumptions on the nature of the MR (Manski, 1989). ACMG treated its MCKAT SS score as a population parameter, rather than the sample estimate that it was. Bootstrapping estimated sampling variability around the estimate based on the sample size and the possible MR. A range of [64.5, 65.5) was consistent with 65% reported (actual score [129, 131)). This was consistent with MR = 0, 3 or 6 out of the 23 responses to any MCKAT question. Boundary estimates were, respectively, [130.4, 130.4], [113,139.1] and [95.7, 147.8]. The last implies the identification region includes a score that would imply a recommendation against screening. Bootstrapping around the original mean showed 5.9 % of bootstraps below the 100 score cutoff. Bootstrapping around the two lower bounds corresponding to MR = 3 and 6 implied 27.9 % and 57.3 % of means below 100. ACMG should have completely reported MR and respondent scores (or variances) for all survey questions. Then readers and policy makers could fully assess the confidence they can have in recommendations. Accounting for uncertainty in this one question indicates a lack of confidence in the recommendation for MCKAT.