BT09 Clinical performance of an artificial intelligence-based medical device deployed within an urgent suspected skin cancer pathway

Abstract

Abstract A secondary care trust received over 2800 urgent suspected skin cancer referrals (USCRs) in 2021, 50% more than in 2016 (www.england.nhs.uk/statistics/statistical-work-areas/cancer-waiting-times), < 1 in 10 of which resulted in diagnosis of skin cancer (www.cancerdata.nhs.uk/cwt_conversion_and_detection). Judicious implementation of artificial intelligence (AI) may improve referral accuracy. We conducted a prospective, postdeployment, single-centre clinical performance review of a UK Conformity Assessed Class IIa-approved, AI-based, skin lesion analysis medical device intended for use in the screening, triage and assessment of skin lesions suspicious for skin cancer (AIaMD). The AIaMD is deployed at the trust as a decision-support tool following USCR from primary care, with a second read of all cases by a consultant dermatologist. Patients assessed by the two most recent versions of AIaMD from October 2021 to October 2022 were eligible for inclusion. Outcomes were confirmed from histology reports for cancerous lesions and histology reports or Consultant teledermatology assessment for noncancerous lesions. The AIaMD assessed 1363 lesions with outcomes confirmed, including 51 melanomas, 64 squamous cell carcinomas (SCCs), 96 basal cell carcinomas, 8 rare skin cancers and 244 premalignancies. The AIaMD referral sensitivity for melanoma, all skin cancer and premalignancy was 100% (n = 51/51), 99.5% (n = 218/219) and 96.3% (n = 235/244), respectively. The AIaMD benign lesion specificity was 55.9% (n = 434/777). The negative predictive value (how often AIaMD labelled lesions as premalignant/benign and correctly ruled out cancer) was 99.8% (n = 539/540). The conversion rate (the percentage of lesions classified as high risk by AIaMD that resulted in a diagnosis of melanoma or SCC) was 13.8% (n = 122/886), while the USCR conversion in the local clinical commissioning group was 9.9% in 2020–2021. The service discharged 9.4% (n = 239/2551) of patients during this time without using trust dermatologist capacity, with the potential to have discharged 18.3% (n = 467/2551) without a second read. This postmarket service evaluation reported the clinical outcomes of an AI device used for assessment of skin lesions. The pathway was sensitive, identifying 218 of 219 skin cancers, and specific, correctly identifying over half of benign lesions. Moreover, AIaMD was able to improve the conversion rate of USCRs. The integration of AIaMD into skin cancer diagnostic pathways could significantly improve the accuracy of USCRs. Conflicts of interest: these were commercial deployments of the AI, and three of the authors are employed by the AI provider.