Rolling bearings are extremely critical rotating mechanical components, and when they fail, they can damage the equipment, causing safety threats or economic losses. Collecting and analyzing vibration signals of faulty bearings is a common fault diagnosis method. Modulation signal bispectrum (MSB) can utilize the inherent modulation characteristics of the signal to achieve fault feature recognition, but it is easily affected by strong background noise and other modulation signals, making it difficult to detect the fault feature frequency in the end. Based on this, a time–frequency modulation bispectrum (TFMB) method is proposed in this paper. The signal is first transformed into the time–frequency domain through short-time Fourier transform, retaining both time and frequency information. It reduces the amplitude of bispectral noise, highlights spectral lines related to fault information, and displays modulation information more clearly, making the demodulation results more accurate. Based on the characteristics of bispectrum, a two-dimensional singular value decomposition (2D SVD) strategy is designed to reduce the amplitude of the noise components in the bispectrum, and the relative change rate is proposed to automatically determine the number of effective singular values. At the same time, considering that the Gini index has good sparse property, the modulation Gini index (MGI) is proposed based on the Gini index, which can identify the repetitive pulses in the signal, suppress the random pulses and noise interference, and extract the modulation component. MGI can be used to evaluate the proportion of fault information contained in TFMB slices and select the optimal frequency slice. The simulation signal verifies the effectiveness of the method in fault diagnosis of rolling bearing, and the proposed method is applied to the diagnosis of rolling bearing outer ring fault, inner ring fault, and compound fault respectively. The experimental results show that the method can effectively reduce noise amplitude, enhance fault information in the bispectrum, and extract the optimal frequency slice to accurately detect bearing fault characteristic frequencies. The proposed method is compared with MSB detector, Fast Kurtogram and Autogram, which proves the superiority of TFMB method in extraction of rolling bearing fault characteristics.