PIH35 - Value and Impact of Incorporating Uncertainty into Opinion Based Research: Study on American College of Medical Genetics (ACMG) Recommendations for Newborn Screening (NBS) for Cystic Fibrosis (CF)

Abstract

ACMG made recommendations for NBS using an algorithm whereby survey scores determined an entry point to an algorithm (EPA). The EPA-specific follow up questions determined the final recommendation for CF as a core panel condition. ACMG largely ignored the influence of missing values and failed to acknowledge uncertainty around survey scoring. The objective of this study is to identify potential number of missing responses (M) from unreported survey responses and to estimate the influence of uncertainty due to missing data and sampling variation on the total disease-specific score and final recommendation made for CF. Questions were analyzed and estimates for potential missing responses were made. The influence of missing values was estimated through boundary estimates (Manski, 1989); bootstrapping estimated the influence of sampling variation. Their joint influence was also examined. The score (90) for one question implied a range of possible missing responses (2-7 of 65 expected responses). Using the conservative M=2, the boundary estimate implied a possible reduction of up to 2.786 points from the original mean (lower bound, LB=87.69). Bootstrapping around the LB indicated 70.2% of means to be lower than the critical value for an EPA and recommendation change. In CF, a loss in score of > 1 point changes the recommendation to “not recommended.” Even with a conservative estimate of missing responses (2) to this one question, missing data could influence the EPA and final recommendations. Adding the influence of sampling variation implies even less confidence around the scoring and recommendation. Our analyses, incorporating adjustments for both types of uncertainty, call into question any confidence in the ACMG screening recommendations even within its own opinion-based process and further show the ambiguity of relying on opinion-based research for policy making, particularly when its apparent precision is revealed as an illusion.