A novel deep learning method for recognizing texts printed with multiple different printing methods

Abstract

Background: Text recognition of cardboard pharmaceutical packages with machine vision is a challenging task due to the different curvatures of packaging surfaces and different printing methods. Methods: In this research, a novel deep learning method based on regions with convolutional neural networks (R-CNN) for recognizing binarized expiration dates and batch codes printed using different printing methods is proposed. The novel method recognizes the characters in the images without the need to extract handcrafted features. In detail, this approach performs text recognition considering the whole image as an input extracting and learning salient character features straight from packaging surface images. Results: The expiration date and manufacturing batch codes of a real-life pharmaceutical packaging image set are recognized with 91.1% precision with a novel deep learning-based model, while Tesseract OCR text recognition performance with the same image set is 38.3%. The novel model outperformed Tesseract OCR also in tests evaluating recall, accuracy, and F-Measure performance. Furthermore, the novel model was evaluated in terms of multi-object recognition accuracy and the number of unrecognized characters, in order to achieve performance values comparable to existing multi-object recognition methods. Conclusions: The results of this study reveal that the novel deep learning method outperforms the well-established optical character recognition method in recognizing texts printed using different printing methods. The novel method presented in the study recognizes texts printed with different printing methods with high precision. The novel deep learning method is suitable for recognizing texts printed on curved surfaces with proper preprocessing. The problem investigated in the study differs from previous research in the field, focusing on the recognition of texts printed with different printing methods. The research thus fills a gap in text recognition that existed in the research of the field. Furthermore, the study presents new ideas that will be utilized in our future research.