Developing an artificial intelligence-based headache diagnostic model from a clinic patients' dataset.

Abstract

We developed an artificial intelligence (AI)-based headache diagnosis model using a large questionnaire database in a headache-specializing clinic. Misdiagnosis of headache disorders is a serious issue and AI-based headache diagnosis models are scarce. We developed an AI-based headache diagnosis model and conducted internal validation based on a retrospective investigation of 6058 patients (4240 training dataset for model development and 1818 test dataset for internal validation) diagnosed by a headache specialist. The ground truth was the diagnosis by the headache specialist. The diagnostic performance of the AI model was evaluated. The dataset included 4829/6058 (79.7%) patients with migraine, 834/6058 (13.8%) with tension-type headache, 78/6058 (1.3%) with trigeminal autonomic cephalalgias, 38/6058 (0.6%) with other primary headache disorders, and 279/6058 (4.6%) with other headaches. The mean (standard deviation) age was 34.7 (14.5) years, and 3986/6058 (65.8%) were female. The model's micro-average accuracy, sensitivity (recall), specificity, precision, and F-values for the test dataset were 93.7%, 84.2%, 84.2%, 96.1%, and 84.2%, respectively. The diagnostic performance for migraine was high, with a sensitivity of 88.8% and c-statistics of 0.89 (95% confidence interval 0.87-0.91). Our AI model demonstrated high diagnostic performance for migraine. If secondary headaches can be ruled out, the model can be a powerful tool for diagnosing migraine; however, further data collection and external validation are required to strengthen the performance, ensure the generalizability in other outpatients, and demonstrate its utility in real-world settings.