During the construction of cable joints, three common defects may occur on the XLPE surface: scratches, moisture exposure, and adhered contaminated particles. To evaluate the impact of these defects on joint performance, this paper establishes a sheet model of XLPE insulation in cable joints to analyze the changes in the electric field under different defects and explore the influence of different defects on the electric field and breakdown voltage. Results of the study reveal that the electric field at the scratch site on XLPE produces severe distortion, being 1.6 times that of non-scratch areas. When exposed to moisture, the more conductive impurities present in the adhered contaminated water on the XLPE surface, the higher the conductivity of the contaminated water, thereby increasing its conductive performance and the electric field strength, which is 1.22–1.4 times that of the non-moist interface. When particles adhere to the XLPE surface, severe distortion occurs at the particle-interface electric field, approximately 1.5 times that of the defect-free interface. Scratches have the most significant impact on the electric field of XLPE insulation. Experimental results also demonstrate that the breakdown voltage without defects is 129.6 kV, while the breakdown voltage with scratch defects is 59.1 kV, moisture defects is 69.7 kV, and particle contamination defects is 59.2 kV, with scratches having the most significant impact on the breakdown voltage of XLPE insulation. These findings provide important insights into the influence of different defects on the insulation performance of cable joints.