Methods and Limitations of Assessing New Noninvasive Tests

Abstract

Outcomes-based approaches are the preferred methodology for technology validation. As discussed in part I of this review, the difficulties of performing an unbiased diagnostic evaluation are increasingly appreciated.1 An outcomes-based approach is advantageous in that it mimics the clinical application of testing. By risk stratifying patients, the results of this approach can be applied directly to clinical practice.2 Nonetheless, outcomes-based technology validation is not without challenges and limitations. These issues include the need for multivariable modeling for observational data, end point selection, and limitations of estimating posttest risk. Finally, it is increasingly appreciated that the future “gold standard” for outcomes-based assessments will be demonstrating whether imaging can identify which therapeutic approach optimizes patient benefit rather than merely identifying risk. ### Study Design Requirements of imaging studies include a relevant study population, comparison with an appropriate control group, and follow-up for outcomes. Designing randomized controlled trials (RCTs) that address imaging questions is challenging. The RCT may utilize imaging results as inclusion criteria or the basis for therapeutic assignment. Randomization to strategies with versus without imaging is problematic. The use of imaging per se affects outcomes only if a therapy is triggered; hence, in these studies, therapy and imaging results must be linked.3,4 Comparisons of imaging methods/modalities must mandate that test results are acted on rather than “available to the physician” (thus, an emphasis on efficacy rather than effectiveness). Because observational studies are far more common, they will be our focus. Although limited by inherent design flaws (eg, selection biases, potentially spurious observations, missing covariates), patients in observational studies better represent those seen in practice. #### Patient Selection Unlike diagnostic-based validation in which only patients referred to a gold standard after testing are included, in prognostic-based approaches all eligible patients are followed up at a preselected time point after testing to determine their status relative …