AB1103 AUTOMATIC FINGER JOINT BONE EROSION SCORE PREDICTION CONSIDERING 2-TIME-POINT X-RAYS OF PATIENTS WITH RHEUMATOID ARTHRITIS BY DEEP LEARNING

Abstract

Background:Medical image analysis using deep learning (DL) has been attracting attention. In previous research, we proposed a DL method for detection of joint region and evaluation for bone destruction at a single point in time in hand X-rays of patients with rheumatoid arthritis (RA) [1-2]. However, in the score of van der Heijde-modified total Sharp scores (mTSS) in X-rays, it is difficult to apply the method as it is. In mTSS, score difference between 2-time points is important, and there is a problem that the score at each time varies depending on the doctor who evaluates.Objectives:We aimed at developing an mTSS scoring method considering 2-time-point difference with a DL method.Methods:A total of 104 X-ray image sets of both hands at two time points with an interval of ≥1 year were randomly obtained from patients with RA who had visited our clinic in 2015. Well-trained doctors determined the erosion scores of MP and PIP/IP joints of each hand in X-rays according to mTSS. These evaluations of hand joints were performed using our developed annotation software tool. In the learning phase, joint images were randomly divided into five sets for 5-fold cross-validation. We utilized a convolutional neural network model, such as SSD [3], for detecting joint regions and classifying the scores (Fig 1).Figure1.The models for classification were designed in consideration of the difference in erosion scores of each patient between the 2-time points of X-rays. The loss function of the DL model was defined bellow;SCE: softmax cross entropyMSE: mean squared errort: training datay: output of DL model0: the former time point1: the latter time pointT: transpose of matrixHere, the coefficient γ is designed to reduce the error for another set of scores with equal differences. The first term of the loss function works to optimize the score at each time point, and the second term works to optimize the score difference at both time points. Thus, our method can be trained without being affected by characteristic training data.Results:The number of joints with differences in erosion score between the former and latter time points was 1 (-2 points), 9 (-1), 2015 (0), 32 (+1), 17 (+2), and 6 (+3). There were no joints with score changes of -5, -4, -3, +4, and +5 points.As a performance of predicting the difference in erosion score between the 2-time points of each patient’s X-ray, our models presented a mean error of 0.412 per each joint in one set for 5-cross validation as compared with physicians’ evaluation (Fig 2).Figure 2.Conclusion:Our DL-based models to predict hand joint erosion scores in X-rays were developed with relatively small samples. This suggests that the predictive performance may increase by collecting more training dataset. Next, we will apply our method to the prediction of joint space narrowing score.