ABSTRACT The Extreme Learning Machine (ELM) has sparked a lot of attention since it can learn fast and be applied to various problems. In this study, a convolutional layer-based extreme learning machine (CELM) architecture has been designed and implemented to recognize handwritten characters and reduce execution time. Furthermore, to validate the robustness of the approach, the characters are chosen from four different languages, mainly Indian languages, including English. The recognition has been performed on both numerals and alphabets. The experimental results regarding accuracy and execution time have been presented in a tabular fashion to discriminate the slight differences between the different datasets chosen. On three different datasets, including chars74kFnt, cMaterDB 3.1.2, and IIITBOdiaV1, the proposed method achieves an accuracy of 91.76%, 94.12%, and 91.43%, respectively. This effectiveness is shown to be higher than other feature extraction method-based recognition methods found in recent experiments.