Response

Abstract

We thank Drs Lam, Mayo, and Tammemagi for their comments on our article.1MacMahon H. Li F. Jiang Y. Armato III, S.G. Accuracy of the Vancouver Lung Cancer Risk Prediction Model compared with radiologists.Chest. 2019; 156: 112-119Abstract Full Text Full Text PDF PubMed Scopus (8) Google Scholar We do not disagree with their assertion that the Vancouver model may play a useful role in determining the need for CT follow-up imaging of small nodules in the context of a screening program, or that future incorporation of radiomic features and artificial intelligence could improve the model’s accuracy. As noted in our article, it was necessary to use an enriched population sample with a relatively high proportion of cancers to perform a meaningful observer test, and we adjusted the model’s output to take this approach into account. We understand that the model was not intended to be prescriptive. However, in practice, a risk model such as this one, which indicates a specific probability of malignancy, will be used in patient management decisions in a variety of circumstances, including consideration of possible intervention, and it has been recommended for estimation of cancer risk in lung nodules.2McWilliams A. Tammemagi M.C. Mayo J.R. et al.Probability of cancer in pulmonary nodules detected on first screening CT.N Engl J Med. 2013; 369: 910-919Crossref PubMed Scopus (792) Google Scholar, 3White C.S. Dharaiya E. Campbell E. Boroczky The Vancouver Lung Cancer Risk Prediction Model: assessment by using a subset of the National Lung Screening Trial Cohort.Radiology. 2017; 283: 264-272Crossref PubMed Scopus (28) Google Scholar Regardless of the authors’ intentions, use of the model is not confined to small low-risk nodules in a screening program. The purpose of our article was to point out the limitations of the model in its current form when applied to larger nodules with more distinctive morphology. Regarding the specific points raised, 18 of the included nodules were subsolid. The 50% cancer risk cut point was chosen as a practical threshold to separate cases with divergent malignant/benign predictions by the model and observers, for comparison in Table 2. We did not suggest that this threshold should be used as a criterion for further evaluation in a screening program. More importantly, this choice of threshold does not affect the receiver-operating characteristic curves on which our conclusions are based, which show that the radiologists outperform the model at all practical thresholds. We do not agree that identifying the nodule with an arrow introduces observer bias in this specific task of individual nodule risk estimation. Regarding Figure 4, different risk estimates can be derived from the model depending on the input parameters. We used the inputs provided by our observers to simulate clinical practice, which indicated the risk as shown. We therefore stand by our conclusion, which is consistent with that of others,4van Riel S.J. Ciompi F. Winkler Wille M.M. et al.Malignancy risk estimation of pulmonary nodules in screening CTs: comparison between a computer model and human observers.PLoS One. 2017; 12e0185032Crossref PubMed Scopus (26) Google Scholar namely that the model in its present form may provide misleading results in an important subset of larger nodules and that a radiologist’s risk estimation is likely to be more accurate in such cases. Accuracy of the Vancouver Lung Cancer Risk Prediction Model Compared With That of RadiologistsCHESTVol. 156Issue 1PreviewRisk models have been developed that include the subject’s pretest risk profile and imaging findings to predict the risk of cancer in an objective way. We assessed the accuracy of the Vancouver Lung Cancer Risk Prediction Model compared with that of trainee and experienced radiologists using a subset of size-matched nodules from the National Lung Screening Trial (NLST). Full-Text PDF Human Observer vs Prediction Model for Lung Nodule Malignancy Risk EstimationCHESTVol. 156Issue 4PreviewWe read with interest the paper by MacMahon et al1 in an issue of CHEST (July 2019) comparing the accuracy of experienced thoracic radiologists and trainees vs the PanCan (Vancouver) Risk Prediction model2 to estimate the likelihood of malignancy of lung nodules found by using screening CT imaging. Full-Text PDF