Abstract
BackgroundDrug label, or packaging insert play a significant role in all the operations from production through drug distribution channels to the end consumer. Image of the label also called Display Panel or label could be used to identify illegal, illicit, unapproved and potentially dangerous drugs. Due to the time-consuming process and high labor cost of investigation, an artificial intelligence-based deep learning model is necessary for fast and accurate identification of the drugs.MethodsIn addition to image-based identification technology, we take advantages of rich text information on the pharmaceutical package insert of drug label images. In this study, we developed the Drug Label Identification through Image and Text embedding model (DLI-IT) to model text-based patterns of historical data for detection of suspicious drugs. In DLI-IT, we first trained a Connectionist Text Proposal Network (CTPN) to crop the raw image into sub-images based on the text. The texts from the cropped sub-images are recognized independently through the Tesseract OCR Engine and combined as one document for each raw image. Finally, we applied universal sentence embedding to transform these documents into vectors and find the most similar reference images to the test image through the cosine similarity.ResultsWe trained the DLI-IT model on 1749 opioid and 2365 non-opioid drug label images. The model was then tested on 300 external opioid drug label images, the result demonstrated our model achieves up-to 88% of the precision in drug label identification, which outperforms previous image-based or text-based identification method by up-to 35% improvement.ConclusionTo conclude, by combining Image and Text embedding analysis under deep learning framework, our DLI-IT approach achieved a competitive performance in advancing drug label identification.
Highlights
Drug label, or packaging insert play a significant role in all the operations from production through drug distribution channels to the end consumer
When a drug is under suspension, it could be evaluated by retrieving drug images from a historical reference dataset for the most similar candidates
Datasets The image samples were collected from Daily-Med; 43% of the images belonged to opioid drugs label and 57% belonged to non-opioid drug labels
Summary
Packaging insert play a significant role in all the operations from production through drug distribution channels to the end consumer. Motivation Drug control and drug distribution play a significant role in providing consumers and health professionals with the products that they need. The investigation process can take time, practically when rejecting a questionable drug product or supplier. To accelerate the process of investigation, the institute must establish and maintain an adequate record in the reference dataset containing images provided by investigators of previously rejected drugs. When a drug is under suspension, it could be evaluated by retrieving drug images from a historical reference dataset for the most similar candidates. Investigators would only need to take an image of a drug’s pharmaceutical packaging and our model would automatically extract text from the image and return the most similar candidates from the historical reference dataset. The investigator could make a quick decision for the necessity of investigation
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
