Tinnitus is a phantom auditory sensation that commonly co-occurs with hearing loss. Both tinnitus and hearing loss can impact the quality of life, emotional well-being, and cognitive functioning of the affected individuals. While previous studies have highlighted structural alterations in hearing loss and/or tinnitus, the fundamental neural mechanisms underpinning tinnitus severity remain poorly understood. In this study, we conducted a voxel-based morphometry to investigate gray matter (GM) volume differences among groups of participants with varying tinnitus severity and hearing status, and controls within a large sample. We observed reduced GM volume in the left anterior insula and right planum polare in participants with hearing loss, regardless of their tinnitus status, compared to normal hearing controls. We noted decreased GM volume in the bilateral anterior and posterior insula for those with tinnitus and normal hearing compared to a normal hearing control group. Further, the tinnitus with hearing loss group showed decreased GM volume in the left planum polare, left inferior temporal gyrus, bilateral anterior temporal gyri, and right superior frontal gyrus compared to the normal hearing control group, suggesting a combined effect of hearing loss and tinnitus. While tinnitus severity did not show a significant overall effect, there was a significant positive correlation between tinnitus distress and GM volume in bilateral planum polare. Our findings enhance the understanding of structural brain changes related to hearing loss and tinnitus, and advance the overall knowledge of tinnitus pathophysiology, which can contribute to the development of more effective treatments for tinnitus.