In a diesel engine, piston slap commonly occurs concurrently with fuel combustion and serves as the main source of excitation. Although combustion pressure can be measured using sensors, determining the slap force is difficult without conducting tests. In this study, we propose a method to identify the slap force of the piston to solve this difficult problem. The traditional VMD algorithm easily receives noise interference, which affects the value of parameter combination [k, α] and thus affects the extraction accuracy of the algorithm. First, we obtain the transfer function between the incentive and vibration response through percussion tests. Secondly, a variational modal decomposition method based on whale algorithm optimization is used to separate the slap response from the surface acceleration of the block. Finally, we calculated the slap force using the deconvolution method. Deconvolution is a typical inverse problem of mathematics, often prone to ill-conditioning, and the singular value decomposition and regularization method is used to overcome this flaw and improve accuracy. The proposed method provides an important means to evaluate the angular distribution of the slap force, identify the shock positions on the piston liner, and determine the peak value of the waveform which helps us analyze the vibration characteristics of the piston and optimize the structural design of the engine.