Due to the inherent unknown image deformation among the training and test samples, performance of the deep convolutional neural network (CNN) will be degraded for Inverse Synthetic Aperture Radar (ISAR) automatic target recognition. Meanwhile, traditional CNN only captures the local spatial information due to small receptive fields, thus, neglects the global information useful for recognition. To tackle these issues, this article proposes the attention-augmented deformation robust ISAR image recognition network, dubbed as ADRISAR-Net. The model adopts the inverse compositional spatial transformer for automatic image deformation adjustment, and performs joint local and global feature extractions by the attention-augmented CNN. Finally, the softmax classifier outputs the recognition results. The proposed ADRISAR-Net is end-to-end trainable, and achieves higher recognition accuracy for the four-satellite and three-airplane ISAR image data sets generated by electromagnetic computing.