Determining ultra-low cycle fatigue (ULCF) life of stainless steel typically involves laborious and time-consuming tests. While machine learning offers an efficient solution for fatigue life prediction, the inherent demand for sufficient training data remains unaddressed. To solve this challenge, a novel method to predict ULCF life using small datasets was proposed by combining artificial neural network (ANN) and transfer learning (TL). A TL-guided ANN (TLNN) framework was developed, wherein the knowledge gained from a pre-trained ANN model for structural steels was transferred to ULCF life prediction of stainless steel. The TLNN model exhibits enhanced predictive capabilities for stainless steel under small amounts of training datasets, with mean R2 (coefficient of determination) value of 0.88 and average MAPE (mean absolute percentage error) value of 25.52%. In comparison, the directly trained ANN model shows lower performance, possessing that the average R2 and MAPE are 0.69 and 47.58%, respectively. We pioneeringly explored and found that adding synthetic data enhancement had no positive effect on the predictive ability of the TLNN model. Furthermore, the dependence of predictive capacity by the TLNN model on the number of training data availability was studied. The prediction process of the TLNN model was interpreted by SHAP (SHapley Additive exPlanations) method. In general, the developed TLNN model can efficiently predict the ULCF life of stainless steel with high accuracy and reduce the cost of consecutive fatigue property evaluation.