Validation of a Market-Approved Artificial Intelligence Mobile Health App for Skin Cancer Screening: A Prospective Multicenter Diagnostic Accuracy Study

Abstract

Background: Mobile health (mHealth) consumer applications (apps) have been integrated with deep learning for skin cancer risk assessments. However, prospective validation of these apps is lacking. Objectives: To identify the diagnostic accuracy of an app integrated with a convolutional neural network for the detection of premalignant and malignant skin lesions. Methods: We performed a prospective multicenter diagnostic accuracy study of a CE-marked mHealth app from January 1 until August 31, 2020, among adult patients with at least one suspicious skin lesion. Skin lesions were assessed by the app on an iOS or Android device after clinical diagnosis and before obtaining histopathology. The app outcome was compared to the histopathological diagnosis, or if not available, the clinical diagnosis by a dermatologist. The primary outcome was the sensitivity and specificity of the app to detect premalignant and malignant skin lesions. Subgroup analyses were conducted for different smartphone types, the lesion’s origin, indication for dermatological consultation, and lesion location. Results: In total, 785 lesions, including 418 suspicious and 367 benign control lesions, among 372 patients (50.8% women) with a median age of 71 years were included. The app performed at an overall 86.9% (95% CI 82.3–90.7) sensitivity and 70.4% (95% CI 66.2–74.3) specificity. The sensitivity was significantly higher on the iOS device compared to the Android device (91.0 vs. 83.0%; p = 0.02). Specificity calculated on benign control lesions was significantly higher than suspicious skin lesions (80.1 vs. 45.5%; p < 0.001). Sensitivity was higher in skin fold areas compared to smooth skin areas (92.9 vs. 84.2%; p = 0.01), while the specificity was higher for lesions in smooth skin areas (72.0 vs. 56.6%; p = 0.02). Conclusion: The diagnostic accuracy of the mHealth app is far from perfect, but is potentially promising to empower patients to self-assess skin lesions before consulting a health care professional. An additional prospective validation study, particularly for suspicious pigmented skin lesions, is warranted. Furthermore, studies investigating mHealth implementation in the lay population are needed to demonstrate the impact on health care systems.