Across the globe, dyslexia and dysgraphia are two frequent learning disorders identified in classrooms. This condition is characterized by difficulties in age-appropriate reading without any sociocultural restrictions. Children with this disorder have difficulty recognizing word and letter patterns. Early identification of dyslexic children (DC) is crucial for providing them with the most effective educational opportunities. Researchers proposed a deep learning-based dyslexia detection system (DDS). However, there is a demand for a practical, lightweight framework for identifying DC. Thus, the proposed study intends to build a framework for detecting dyslexia. The proposed framework encompasses image processing, feature extraction, and classification models. The image-processing model enhances the image quality using contrast-limited adaptive histogram equalization and resizes the images into 512 × 512 pixels. For feature extraction, the authors employ you only look once V7 to extract features in a limited time. In addition, the MobileNet V2 with single shot detection lite is used to classify the handwritten images into normal and abnormal classes, respectively. The authors utilized the publicly available dyslexia dataset for performance evaluation. The test set contains 19,557 normal and 17,882 reversal (abnormal) images. The baseline models are employed for comparative analysis. The experimental study revealed that the proposed framework outperformed the baseline models by achieving exceptional precision, recall, F1-Score, accuracy, and mean average precision of 97.9, 97.3, 97.6, 99.2, and 97.6, respectively. In addition, the proposed model obtained an exceptional mean intersection over union of 88.6. It can be implemented in educational institutions and healthcare centers. In the future, the authors can extend the research to build an integrated framework using biomedical images.