A representative general polarimetric model-based decomposition framework was proposed by Chen et al., which implements a simultaneous full-parameter inversion by using complete polarimetric information and solves several limitations in previous decomposition methods. However, there are still shortcomings in Chen’s work. Firstly, only the real part of the parameter β in the generalized surface scattering model is considered. Secondly, inappropriate initial input values may lead to local optima in the nonlinear least squares optimization algorithm. Thirdly, the volume scattering component is underestimated in the volume scattering-dominated scene, but overestimated in buildings with large orientation (LOB) areas. Finally, nonlinear optimization is time-consuming computationally. To overcome those issues, an improved generalized polarimetric model-based decomposition method is proposed in this paper. The imaginary part of the parameter β is incorporated into the decomposition framework of the proposed method. Ingeniously utilizing the internal relationship in the generic equations composed of coherent matrix elements, the model parameters can be inversed by simplifying the nonlinear equations to linear equations. Therefore, compared with Chen’s method, the proposed method does not rely on the initial input values, and improves the computational efficiency. In addition, a hierarchical decomposition scheme is presented to solve the problem of underestimation or overestimation of volume scattering component mentioned above. The performance and advantages of this method are evaluated with L-band and C-band polarimetric synthetic aperture radar (PolSAR) data sets. Comparison studies are carried out with other model-based decomposition methods, demonstrating that the proposed method can further improve decomposition performance, especially in LOB areas.