There was an investigation of the diagnostic and prognostic effect of magnetic resonance imaging (MRI) based on multimodal feature fusion algorithm for impotence of perianal abscess. In this study, the second to fifth convolution blocks of the visual geometric group network were applied to extract the depth features in the way of transfer learning, and a multimode feature fusion algorithm was constructed. The whole network was trained by maximizing the energy proportion of the feature layers, which was compared with the fully convolutional neural network (FCN) algorithm. Then, this algorithm was adopted to the imaging diagnosis of 50 patients with anorectal diseases admitted to our hospital, and it was found that the similarity coefficient (85.37%), accuracy (80.02%), and recall rate (79.38%) of the improved deep learning algorithm were higher markedly than those of the FCN algorithm (70.18%, 67.82%, and 66.92%) (P < 0.05). As the number of convolutional layers increased, the segmentation accuracy of the convolutional neural network (CNN) algorithm was also improved. The detection rate of the observation group (84%) rose hugely compared with the control group (64%), and the difference was statistically obvious (P < 0.05). Besides, the detection accuracy of abscess location (84%), impotent tract location (80%), and internal orifice location (92%) in patients from the observation group was higher substantially than the accuracy of abscess location (60%), impotent tract location (68%), and internal orifice location (72%) from the control group (P < 0.05). In conclusion, the performance of the multimodal feature fusion algorithm was better, and the MRI image feature analysis based on this algorithm had a higher diagnostic accuracy, which had a positive effect on improving the detection rate, detection accuracy, and disease classification.