Artificial Intelligence-Based Disease Activity Monitoring to Personalized Neovascular Age-Related Macular Degeneration Treatment: A Feasibility Study

Abstract

PurposeTo evaluate the performance of a disease activity (DA) model developed to detect DA in participants with neovascular age-related macular degeneration (nAMD). DesignPost-hoc analysis. ParticipantsPatient dataset from the phase III HAWK and HARRIER (H&H) studies. MethodsAn artificial intelligence (AI)-based DA model was developed to generate a DA score based on measurements of optical coherence tomography (OCT) images and other parameters collected from H&H study participants. DA assessments were classified into three categories based on the extent of agreement between the DA model’s scores and the H&H investigators’ decisions: agreement (“easy”), disagreement (“noisy”), and close to the decision boundary (“difficult”). Then, a panel of 10 international retina specialists (“panelists”) reviewed a sample of DA assessments of these three categories that contributed to the training of the final DA model. A panelists’ majority vote on the reviewed cases was used to evaluate the accuracy, sensitivity, and specificity of the DA model. Main Outcome MeasureThe DA model’s performance in detecting DA compared with the DA assessments made by the investigators and panelists’ majority vote. ResultsA total of 4472 OCT DA assessments were used to develop the model; of these, panelists reviewed 425, categorized as “easy” (17.2%), “noisy” (20.5%), and “difficult” (62.4%). False-positive and false-negative rates of the DA model’s assessments decreased after changing the assessment in some cases reviewed by the panelists and retraining the DA model. Overall, the DA model achieved 80% accuracy. For “easy” cases, the DA model reached 96% accuracy and performed as well as the investigators (96% accuracy) and panelists (90% accuracy). For “noisy” cases, the DA model performed similarly to panelists and outperformed the investigators (84%, 86%, and 16% accuracies, respectively). The DA model also outperformed the investigators for “difficult” cases (74% and 53% accuracies, respectively), but underperformed the panelists (86% accuracy) owing to lower specificity. Subretinal and intraretinal fluids were the main clinical parameters driving the DA assessments made by the panelists. ConclusionsThese results demonstrate the potential of using an AI-based DA model to optimize treatment decisions in the clinical setting and in detecting and monitoring DA in nAMD patients.