Smart Colposcopy is a low-cost and highly effective screening device used to detect cervical cancer. However, the captured images can be significantly affected by specular reflection, resulting in reduced accuracy during the grading process. In this paper, proposed a novel approach to enhance the accuracy of cervical cancer grading by addressing the issue of specular reflection. Here the method employs binary masking to identify the glare region and fine-tuned U-Net model to perform segmentation. Partial convolutional inpainting is used to replace the segmented region with neighboring pixels, effectively removing the glare from the images. The resulting enhanced images are then fed into classification models, including DenseNet121, Vgg19, and Efficient Net, which are trained to accurately grade cervical cancer. Experimental results demonstrate that our proposed method significantly improves the accuracy of cervical cancer grading, achieving accuracy rates of 97.32%, 96.25%, and 96.75%, respectively.