The generalized Maxwell (GM) model is widely used to describe the viscoelastic behavior and predict performance of asphalt mixtures. However, the existing methods may not provide satisfactory accuracy because relaxation spectral distribution features are not considered. In this study, the methods (area equalization (AE) and peak equalization (PE) method) consider the distribution characteristics of the relaxation spectrum were proposed to obtain the Prony series parameters of the GM model. Two typical asphalt mixtures, HMA (hot mix asphalt mixture) and WMA (warm mix asphalt mixture), were used as case studies. The Prony series parameters of the GM model obtained by the proposed methods and existing methods (traditional fitting (TF) method and Limit equalization (LE) method) were discussed and analyzed. The derived Prony series parameters were also employed in finite element (FE) simulations to inversely predict dynamic modulus, and the simulated results were then compared with the experimental results. The results indicate that the spring stiffnesses of Maxwell elements in the Prony series parameters obtained by different methods have different distribution features. For the TF method, the Maxwell element with larger relaxation time has smaller spring stiffness. In the LE and PE method, the spring stiffness of the Maxwell element first increases and then decreases as the relaxation time increases. The spring stiffness is the same for the different Maxwell elements in the AE method. More importantly, the number of branches of the GM model is increased from 6 to 12, the relative error of the predicted dynamic modulus of HMA by the PE, AE, LE and TF method is reduced from 29.8% to 4.8%, from 49.9% to 25.9%, from 61.4% to 32.7%, from 159.1% to 133.9% respectively. The PE method has the best accuracy in FE analysis, even with a smaller number of branches of the GM model. The similar conclusion can also be summarized in the dynamic modulus prediction of WMA. Therefore, the PE method that considers the rate of change of the relaxation spectrum is preferentially recommended to obtain the Prony series parameters of the GM model.