СВЕРТОЧНАЯ НЕЙРОСЕТЕВАЯ МОДЕЛЬ РАСПОЗНАВАНИЯ РУКОПИСНЫХ МАТЕМАТИЧЕСКИХ ВЫРАЖЕНИЙ

Abstract

Listen

Translate article

This article is devoted to solving the problem of developing a convolutional neural network model for recognizing handwritten mathematical expressions consisting of Arabic numerals and basic symbols of mathematical operations. Its solution required the formation of training and test samples, development of the structure of the neural network and its training, selection of a method for segmenting the image into individual characters, as well as testing and evaluating the results of the convolutional neural network model. In the work, image processing and classification was carried out using a multilayer convolutional neural network. To build a convolutional neural network model, the deep learning library Keras was used in this work. To train the model, a set of handwritten mathematical symbols was used, obtained using the publicly available machine learning library FastAI and consisting of 10,379 images of handwritten mathematical expressions. To test the equation classifier, a random expression generator was created based on the initial data of the handwritten training sample. Examples of generated mathematical expressions are shown. To simplify the learning process, all images were converted to binary format, saving it in black and white. Thus, image processing required the steps of generating data, receiving input images, scaling them, building an architecture and training a convolutional neural network, as well as its practical use. The result of image processing is to obtain a table of probabilities that the image belongs to a particular class of numbers and expressions of mathematical operations. To divide the images into different areas, a segmentation process was carried out using contour analysis methods. The OpenCV library was used to detect the outlines of numbers and mathematical symbols. Image segmentation required the steps of converting the gray image to binary format, finding contours using the findContours function, and checking for contour intersections. After detecting objects in the image, elements are extracted from the mathematical expression, analyzed using a trained neural network, and the string is digitized. The constructed neural network model showed high accuracy of image recognition of mathematical expressions on training and test data samples. The recognition accuracy was no less than 98%.