RESEARCH OF THE JOINT USE OF MATHEMATICAL MORPHOLOGY AND CONVOLUTIONAL NEURAL NETWORKS FOR THE SOLUTION OF THE PRICE TAG RECOGNITION PROBLEM

Abstract

The work is devoted to solving the problem of recognizing images containing symbolic type information, barcodes, logos and other signs. Example of such images are price tags in shopping centers, flyers, invitations, tickets to various events. The information on such images is of a different type and its recognition requires various approaches. The work addressed the recognition of price tags in retail chains. The accuracy of object detection has a significant role for their recognition. A significant role for recognition of image elements has the accuracy of their detection. The combination of classical methods of image analysis and the neural network approach were investigated. Particular attention was paid to the study in the comparative aspect of the object detection by morphological method and by processing a convolutional neural network. Studies have shown that morphology yields a significantly lower detection quality than a neural network, but is several times faster than it. Since speed has a great importance for the implementation of algorithms on mobile devices, post-processing with additional filters and normalization of geometric distortions were added to the morphology, that significantly improved the accuracy of detection and subsequent recognition. Based on the research results of detection and recognition of barcodes and symbolic information presented on price tags, conclusions are drawn about choosing approaches and technologies for solving these problems, an algorithm has been developed and, on its basis, an application for recognizing price tags of various retail chains. A mobile version of the application has been developed also. The algorithm is constructed in such a way that the first step is the detection of the supporting element, for example, a barcode, then other price tag elements are detected relative to this supporting element. The barcode is detected with the methods based on mathematical morphology and mathematical statistics which were used to improve the accuracy of the algorithm, or convolutional neural networks. To detect prices and product names, the convolutional neural network CRAFT is exploited, which can process low-quality images. The found name and price are normalized to eliminate geometric distortions and transferred to the Tesseract library for recognition. This library works with many languages and is in the public access. The price tag recognition application was created in C ++ using the OpenCV, ZXing, Libtorch, Tesseract libraries.