Rapid and accurate identification of moving load is crucial for bridge operation management and early warning of overload events. However, it is hard to obtain them rapidly via traditional machine learning methods, due to their massive model parameters and complex network structure. To this end, this paper proposes a novel method to perform moving loads identification using MobileNetV2 and transfer learning. Specifically, the dynamic responses of a vehicle–bridge interaction system are firstly transformed into a two-dimensional time-frequency image by continuous wavelet transform to construct the database. Secondly, a pre-trained MobileNetV2 model based on ImageNet is transferred to the moving load identification task by transfer learning strategy for describing the mapping relationship between structural response and these specified moving loads. Then, load identification can be performed through inputting bridge responses into the established relationship. Finally, the effectiveness of the method is verified by numerical simulation. The results show that it can accurately identify the vehicle weight, vehicle speed information, and presents excellent strong robustness. In addition, MobileNetV2 has faster identification speed and requires less computational resources than several traditional deep convolutional neural network models in moving load identification, which can provide a novel idea for the rapid identification of moving loads.