Diabetic retinopathy becomes an increasingly popular cause of vision loss in diabetic patients. Deep learning has recently received attention as one of the most popular methods for boosting performance in a range of sectors, including medical image analysis and classification. The proposed system comprises three steps; they are image preprocessing, image segmentation, and classification. In preprocessing, the image will be resized, denoising the image and enhancing the contrast of the image which is used for further processing. The lesion region of diabetic retinopathy fundus image is segmented by using Feature Fusion-based U-Net architecture. A blood vessel of a retinal image is extracted by using the spatial fuzzy c means clustering (SFCM) algorithm. Finally, the diabetic retinopathy images are classified using a modified capsule network. The convolution and primary capsule layers collect features from fundus images, while the class capsule and softmax layers decide whether the image belongs to a certain class. Using the Messidor dataset, the proposed system’s network efficiency is evaluated in terms of four performance indicators. The modified contrast limited adaptive histogram equalization technique enhanced the Peak Signal to Noise Ratio (PSNR), mean square error, and Structural Similarity Index Measure (SSIM) have average values of 36.18, 6.15, and 0.95, respectively. After enhancing the image, segmentation is performed to segment the vessel and lesion region. The segmentation accuracy is measured for the proposed segmentation algorithm by using two metrics namely intersection over union (IoU) and Dice similarity coefficient. Then modified capsule network is constructed for classifying the stages of diabetic retinopathy. The experimental result shows that the proposed modified capsule network got 98.57% of classification accuracy.