Experience in using artificial intelligence services for diagnosing compression fracture of vertebral body based on computed tomography – from testing to trials

Abstract

Backgraund:Osteoporosis (OP) is often diagnosed at the stage of complications. Compression fractures (CF) of the vertebral bodies, complications of OP, and predictors of subsequent fractures are often asymptomatic. CF can be detected on CT scan performed for other indications. We have analyzed approaches to using artificial intelligence services (AI services) generated for diagnosing CF of the vertebral bodies. Aims:Testing AI services to identify CF based on chest CT and to assess the possibility of their implementation into the practice of medical facilities of the Moscow Healthcare Department. Materials and methods:For setting a clinical task for AI services, basic diagnostic requirements in the direction of “Osteoporosis” are formed. AI services go through the following stages: self-testing, functional and calibration testing, trial and trial operation. The first three stages of testing are carried out on previously generated datasets. At the stages of trial and trial operation, AI services analyze CT examinations performed in medical facilities. An expert group of radiologists evaluates the diagnostic accuracy and functional capacity of AI services at all stages. The obtained quantitative metrics of the accuracy of AI services are compared with the target values ​​specified in the requirements. Results:From June 2021 to June 2022, two AI services (AI service №1 and AI service №2) which used different methods for identifying a presence of CF were tested. AI services successfully passed the self-testing stage (6), functional (5) and calibration (100) testing. The ROC AUC for AI service №1 was 0.99 and for AI service №2–0.91. The AI service №1 passed the trial stage without any significant remarks, while AI service №2 was sent for revision. After the trial operation stage, the following accuracy metrics were obtained: the ROC AUC for AI service №1 was 0.93; for AI service №2–0.92. At all stages, both AI services showed sufficient metrics for clinical validation. Conclusions:A testing of the performance of AI services in the direction of “Osteoporosis” was carried out. A high quality of diagnosis was demonstrated. AI services can act as an auxiliary tool in the medical decision support system.