Bluff-body swirling flows have been widely employed in gas-turbine combustors to achieve flame stabilization. Meanwhile, considerable efforts have been made to understand swirling flow dynamics, the effects of swirl number and bluff body on flow structure and dynamics are still not well understood. To this end, a series of direct numerical simulations of isothermal swirling flows have been conducted in this work in order to investigate the impact of swirl numbers and bluff-body diameters on the flow structure, Reynolds stresses, and turbulent kinetic energy (TKE) transport. It is found that a change in the swirl number can affect the inner recirculation zone (IRZ) and hence momentum transport. Specifically, as the swirl number increases, the vortex core formed at downstream locations can merge with the IRZ. Moreover, including the bluff body not only contributes to the formation of the IRZ but also serves as a disturbance source for the flow, which is favorable for the formation of large-scale vortex structures. Then, the impact of swirl number and bluff body on Reynolds shear stresses and anisotropy invariants is investigated to identify the locations of the inter shear layer (ISL), the outer shear layer (OSL), and the main swirling zone (MSZ). The results show that as the swirl number increases, both the ISL and MSZ shift to the wall, indicating a large IRZ. Furthermore, the analysis of TKE indicates that for cases with a bluff body, TKE mainly occurs in the ISL and OSL, featuring a dual peak distribution. However, for cases without a bluff body, the distribution of TKE is primarily concentrated in the ISL. These results suggest that both increasing the swirl number and/or including the bluff body could help with TKE transport, which can lead to a wide range of TKE distribution.