The material of ACSR in transmission line is prone to local damage, which leads to broken strand defect and reducing power consumption safety. Therefore, an automatic recognition method of broken strand defect image of transmission line based on deep transfer learning is designed to improve the automatic recognition effect of broken strand defect image. The multi-scale algorithm is used to enhance the image. In the feature extraction part of the depth transfer learning framework in the confusion domain, the multi-source domain transfer and dual flow fusion algorithm are used to extract the features of the enhanced image, and the Euclidean distance between the feature vector and the template image feature vector is used to correct the image features; using the corrected image feature training network propagated to the automatic defect recognition part and the domain classification part, the loss function and back propagation algorithm are used to reduce the loss of feature extraction and automatic defect recognition part, and the optimal results of automatic defect recognition and domain classification are obtained. The experimental results show that the method can enhance the image effectively with high definition. At different image angles, the recognition accuracy of this method is as high as 0.96, which has better automatic recognition effect of defect image.