Bengali handwritten equation solving system

Abstract

The goal of this research is to detect, recognize and solve mathematical equations, especially Bangla handwritten equations using a deep learning-based model and JUBHEC (Juel & Utpol’s Bengali Handwritten Equation Converter) method. We are going to suggest a merged model (region-based Convolutional Neural Network (mask-R-CNN) and Feature Pyramid Network (FPN) i.e. Bengali Handwritten Equation Detector (BHED)) to detect and recognize Bengali handwritten equations and the JUBHEC method to convert the detected Bengali handwritten equation into computerized form. We have also used BODMAS (Bracket, Of, Division, Multiplication, Addition, Subtraction) principles and different data structure techniques to solve Bangla handwritten equations. We have also introduced the Bengali Handwritten Arithmetic Equations Dataset (BHAED), which has 1000 images, 23 classes, and 28954 instances, and created polygon shape ground truth for the BHAED dataset. The system has given 99.2% accuracy for fast-RCNN character classification, 98% accuracy for fast-RCNN foreground box classification and 96% accuracy for mask-RCNN instance segmentation of characters.