Software tool for recognition of Ukrainian-language scientific articles

Abstract

The problem of computer recognition, both separately printed characters and whole texts, which may contain mathematical formulas, and further saving the resulting document in the "Latex" format, is considered. The developed software implements the ability to recognize printable Latin, Cyrillic, Greek letters and special mathematical symbols. For this, a multilayer convolutional neural network built using the Keras machine learning library and additional validation heuristics are used. To improve the quality of neural network recognition, a sophisticated image processing mechanism has been developed that helps to remove noise from the image, eliminate errors associated with the inclination of characters, and correct character defects associated with the quality of the input image. Also implemented are mechanisms for collecting individual characters into words or mathematical formulas, reproducing the position of signs of indices and degrees, forming ordinary fractions and expressions under the root sign. The results of the recognized text are saved in a file with the simultaneous construction of the "latex" document structure. To demonstrate the capabilities of the developed software, a graphical user interface has been added, with which you can select and inspect the input image even before the start of recognition. During testing of the software, the recognition of images of different types was carried out: completely textual, mathematical formulas without text, mathematical formulas that are between blocks of text.