Fast and Accurate Ophthalmic Medication Bottle Identification Using Deep Learning on a Smartphone Device

Abstract

To assess the accuracy and efficacy of deep learning models, specifically convolutional neural networks (CNNs), to identify glaucoma medication bottles. Algorithm development for predicting ophthalmic medication bottles using a large mobile image-based dataset. A total of 3750 mobile images of 5 ophthalmic medication bottles were included: brimonidine tartrate, dorzolamide-timolol, latanoprost, prednisolone acetate, and moxifloxacin. Seven CNN models were initially pretrained on a large-scale image database and subsequently retrained to classify 5 commonly prescribed topical ophthalmic medications using a training dataset of 2250 mobile-phone captured images. The retrained CNN models' accuracies were compared using k-fold cross-validation (k= 10). The top 2 performing CNN models were then embedded into separate iOS apps and evaluated using 1500 mobile images not included in the training dataset. Prediction accuracy, image processing time. Of the 7 CNN architectures, MobileNet v2 yielded the highest k-fold cross-validation accuracy of 0.974 (95% confidence interval [CI], 0.966-0.980) and the shortest average image processing time at 3.45 (95% CI, 3.13-3.77) sec/image. ResNet V2 had the second highest accuracy of 0.961 (95% CI, 0.952-0.969). When the 2 app-embedded CNNs were compared, in terms of accuracy, MobileNet V2, with an image prediction accuracy of 0.86 (95% CI, 0.84-0.88), was significantly greater than ResNet V2, 0.68 (95% CI, 0.66-0.71) (Table1). Sensitivities and specificities varied between medications (Table1). There was no significant difference inaverage imaging processing time, 0.32 (95% CI, 0.28-0.36) sec/image and 0.31 (95% CI, 0.29-0.33) sec/image for MobileNet V2 and ResNet V2, respectively. Information on beta-testing of the iOS app can be found here: https://lin.hs.uci.edu/research/. We have retrained MobileNet V2 to accurately identify ophthalmic medication bottles and demonstrated that this neural network can operate in a smartphone environment. This work serves as a proof-of-concept for the production of a CNN-based smartphone application to empower patients by decreasing risk for error.