This article proposes a new fast solution algorithm (IGFBM-SAA), which combines the Improved Generalized Forward and Backward Method (IGFBM) with Spectral Acceleration Approach (SAA), which can effectively solve the electromagnetic scattering problem of layered rough surface. In this article, the electric field integral equations (EFIE) for layered rough surfaces is established, and the traditional forward and backward method (FBM) is introduced. Then, based on the traditional FBM algorithm, an Improved Generalized Forward and Backward Method is proposed and, by using the SAA technique in its iterative process, the computation of matrix-vector multiplication is accelerated, thus enabling rapid solution. In the algorithm validation, the same rough surface was calculated using the MoM, FBM, and IGFBM-SAA. The study found that when root mean square (RMS) heights are h1=h2=0.1λ, the convergence accuracy can reach τ=10−7 after 14 iterations. However, as the roughness increases to h1=h2=0.3λ and h1=h2=0.5λ, the convergence accuracy falling to τ=10−5 and τ=10−5, respectively. This indicates that it is necessary to adjust the integration parameters to improve the convergence accuracy. In addition, it was found that when the size of the rough surface is 25.6λ, the computational times for calculations are 91 s (IGFBM-SAA), 197 s (FBM), and 410 s (MoM), respectively. When the size of the rough surface increases to 51.2λ, the computational time differences become more significant, with 236 s, 756 s, and 2547 s being the respective values. This indicates that the proposed algorithm in this article has significant computational speed advantages when dealing with larger rough surfaces. Based on this algorithm, this article studied the electromagnetic scattering characteristics of layered rough surfaces with different parameters (RMS height, dielectric constant, and correlation length), and relevant research results can provide valuable references for areas such as radar target recognition and radar stealth technology, thereby enhancing the accuracy and reliability of radar detection as well as radar stealthperformance.