Detection of bolt loosening using vibro-acoustic modulation (VAM) has been increasingly investigated in the past decade. However, conventional nonlinear coefficients, derived from theoretical analysis, are usually based on the assumption of ideal wave-surface interactions at the joint interfaces. Such coefficients show a poor correlation with the tightening torque when the joint is under the combined influences from structural and material nonlinearities. A reliable inspection method of residual bolt torque was proposed in this study using Support Vector Regression (SVR) with acoustic features from VAM. By considering material intrinsic nonlinearity (MIN) and dissipative nonlinearity (DN), responses of aluminum-aluminum and composite-composite bolted joints during the VAM test were accurately simulated. SVR were subsequently established based on the database built by a mixing of simulated and experimental nonlinear spectral features when the joints were inspected at different scenarios. The results show that the evaluation of residual torque using the SVR models driven by the acoustic nonlinear responses show a higher accuracy compared with the conventional nonlinear coefficients. Requiring limited experiment data, the proposed method can achieve reliable inspection of bolt torque by including the simulated data into machine training.