To effectively classify inverse synthetic aperture radar (ISAR) sequential image with unknown deformation, a sequential adjustment ISAR image classification network (SAISAR-Net) is proposed, which first performs global and local image adjustments for each image frame and obtains deformation robust feature sequence. Then, the time-varying features are extracted by attention augmented bidirectional long short-term memory (Bi-LSTM), the output of which is weighted and fused to give a classification label. Compared with the existing deep learning methods, the proposed network significantly improves the classification accuracy and exhibits robustness in scenarios of scaled, rotated, combined transformation, and practical satellite orbit tests.