A generalized ensemble approach based on transfer learning for Braille character recognition

Abstract

Braille character recognition(BCR) is a basic step in building and designing any Braille assistive technology. Each Braille character is represented by a 2 × 3 matrix of raised dots (called a cell), which can be read by touch. This study introduces a generalized recognition approach based on an ensemble of transfer learning models for BCR. The study experiments are performed on two benchmark English Braille datasets (handwritten Braille – Omniglot (HBO), and Braille character (BC)), and a new dataset of Arabic Braille characters collected by our group called Arabic Braille (AB). First, we investigate the performance of 17- transfer learning models on the three datasets. Then, we build three ensemble approaches based on majority voting from the most effective two, three, and four models in each dataset. The experimental results reveal that the ensemble of DarkNet-53, GoogleNet, SqueezeNet, and DenseNet-201 is a more generalizable ensemble approach for BCR. It achieves a higher F1 score and lesser generalization error (Etest) value than each individual transfer learning model. The F1 scores of the introduced ensemble reached 89.42%, 99.58%, and 97.11% on the HBO, BC, and AB datasets, respectively, with Etest values of 10.47%, 0.43%, and 3.23%. While the F1 scores of the DarkNet-53 which is the most effective single model on the three datasets are 87.54%, 99.14%, and 94.73, with Etest values of 12.79%, 0.85%, and 5.31%, respectively.