Structural damping, which has been widely used to quantify the energy dissipation of engineering structures, plays an important role in dynamic analysis during the design phase and the assessment of existing structures. However, the development of an accurate damping model remains an open question due to the unclear mechanism. In this study, structural damping was identified based on loss factors for beam structures, which were used for presenting the damping behavior during vibrations. The equation of motion for the bending deformation of a Timoshenko beam was derived, accounting for the material damping. To identify the loss factor of the beam from the response signals at a limited number of measurement points, an inverse algorithm based on wave coefficient estimation was developed, considering the numerical stability for data interpretation. Based on the proposed loss factor identification method, several beam models with different shapes of cross-sections were investigated, focusing on the feasibility of the method of varying cross-section cases. Comparing the spectral element model with the loss factor and the finite element model with Rayleigh damping, it was confirmed that the loss factor can better describe the actual energy dissipation behavior of the structure. It provides a more accurate theoretical model for estimating structural damping in practical engineering.