POS0112 CLASSIFYING SELF-REPORTED RHEUMATOID ARTHRITIS FLARES USING DAILY PATIENT-GENERATED DATA

Abstract

BackgroundFlares are an inherent part of the rheumatoid arthritis (RA) disease course and may impact clinical and patient outcomes. The ability to predict flares between clinic visits based on real-time longitudinal patient-generated data could potentially allow for timely interventions to avoid disease worsening. For intensively-collected patient-generated data, machine learning methods offer benefits over traditional statistical tools for accurate prediction, but examples in rheumatology are sparse.ObjectivesInvestigate the feasibility of using machine learning methods to classify self-reported RA flares based on a small dataset of daily symptom data collected on a smartphone app.MethodsWe used data from the Remote Monitoring of Rheumatoid Arthritis (REMORA) study, which aimed to improve monitoring of disease severity in RA. Patients tracked daily symptoms (pain, fatigue, function, sleep, coping, physical and emotional wellbeing) on a 0-10 numerical rating scale, duration of morning stiffness, and weekly flares on the REMORA smartphone app for three months.The outcome was the binary yes/no answer to the weekly flare question “Have you experienced a flare in the last week?”. Several summaries of the eight daily symptom scores collected in the week leading up to the flare question (the exposure period) were used as predictors. These included the mean, min, max, standard deviation and slope. Where exposure periods overlapped, the intersecting symptom reports were allowed to correspond to multiple outcomes.We fitted three binary classifiers: logistic regression +/- elastic net regularization, a random forest and naïve Bayes. The models were benchmarked using the R package mlr3 and 10-fold cross-validation, with two participants comprising the test set and the remaining 18 the training set.Finally, the performance of the classifiers was evaluated according to the area under the curve (AUC) of the receiver operating characteristic curve. The model with the highest AUC in the test dataset was considered as the best final model.ResultsTwenty patients tracked daily symptoms over three months. 60% were female, all but one were white British, and mean age was 56.9±11.1 years. The median number of days in the study was 81 (interquartile range (IQR) 80, 82). The collected data comprised an average of 60.6 daily reports and 10.5 weekly reports per participant over the study period. Participants reported a median of 2 flares (IQR 0.75-4.25) resulting in 57 flares in total.Classifier performances are visualized in Figure 1. The best performing model was logistic regression with elastic net with an AUC of 0.82. At a cut-off point requiring specificity to be 0.80, the corresponding sensitivity to detect flares was 0.60 for this model, meaning that the prediction model correctly identified three in every five self-reported flares, and four in every five non-flares. At this cut-off, the positive predictive value, i.e. the probability that those with a predicted flare indeed go on to have a flare was 53%. The negative predictive value, i.e. the probability that those with a predicted non-flare indeed do not experience a flare, was 85%.ConclusionPredicting self-reported flares based on daily symptom scorings in the preceding week using machine learning methods was feasible, although regularized logistic regression outperformed the other machine learning methods in this small dataset. The perceived advantage of machine learning may therefore be attributed to overfitting. It is possible that the observed predictive accuracy will improve as we obtain more data.Our results point to a future where regular analysis of frequently collected patient-generated data may allow us to predict imminent flares before they unfold with decent accuracy, opening up opportunities for just-in-time adaptive interventions (JITAIs). Depending on the nature and implications of a JITAI, different cut-off values should be explored: different interventions will require different levels of predictive certainty before an action is triggered (eg self-management advice vs. a patient contact).Disclosure of InterestsJulie Gandrup: None declared, David A Selby: None declared, William Dixon Consultant of: Received consultancy fees from Abbvie and Google