Fatigue crack detection is an important issue to ensure the safety of steel strands. The key to solve this problem is to extract the nonlinear response generated by fatigue cracks. In this paper, the nonlinear VAM method is used to detect the structural cracks. The structure with fatigue cracks is excited using low-frequency pumping and high-frequency probing, and the power spectrum analysis of the modulated signal is carried out. In view of the shortcomings of spectral analysis, a new method combining S-transform and bispectrum is proposed, which is called S-transform bispectrum. S transform contains the phase factor, which can retain the absolute phase characteristics of each frequency, and has good time–frequency multi-scale focusing performance. The bispectrum can suppress Gaussian noise, retain phase information, and quantitatively describe the quadratic phase coupling in the signal. Then the simulation and experiment of damaged straight rod, helical rod, and steel strands are carried out. The results show that the proposed method can effectively detect the nonlinear features by using the sideband peaks in the S-transform bispectrum three-dimensional plot, and the nonlinear features are important to identify the structure with damage. At the same time, in order to prove the ability of S-transform bispectrum, an S-transform bispectrum detector is used to verify it, which is superior to the spectrum in terms of its ability to localize modulation sidebands. The proposed S-transform bispectrum has a good application prospect and provides a new tool for structural damage detection.