Abstract Aiming at the problems of complex diesel engine cylinder head signals, difficulty in extracting fault information, and existing deep learning fault diagnosis algorithms with many training parameters, high time cost, and high data volume requirements, a small-sample transfer learning fault diagnosis algorithm is proposed in this article. First, the fault vibration signal of the diesel engine is converted into a three-channel red green blue (RGB) short-time Fourier transform time–frequency diagram, which reduces the randomness of artificially extracted features. Then, for the problem of slow network training and large sample size requirements, the AlexNet convolutional network and the ResNet-18 convolutional network are fine-tuned on the diesel engine time–frequency map samples as pre-training models with the transfer diagnosis strategy. In addition, to improve the training effect of the network, a surrogate model is introduced to autonomously optimize the hyperparameters of the network. Experiments show that, when compared to other commonly used methods, the transfer fault diagnosis algorithm proposed in this article can obtain high classification accuracy in the diagnosis of diesel engines while maintaining very stable performance under the condition of small samples.