Although Klinefelter syndrome (KS) is the most frequent sex-hormone disorder, there is ongoing uncertainty about the often associated sex-hormone deficiency, its impact on common comorbidities, and therefore about prevention and treatment. In this study, we focus on bone loss, reported to occur in over 40% of KS patients, and the impact of testosterone replacement therapy (TRT) on this comorbidity. This single-center retrospective cohort study in a tertiary hospital compared the effect of treatment with TRT to no TRT on evolution of bone mineral density (BMD) in KS patients. After a medical chart review, a total of 52 KS subjects were included in this study. BMD was measured by dual-energy X-ray absorptiometry (DXA) and expressed as T-scores. The subjects were divided into three groups, according to TRT. In the subgroup that only started TRT after baseline measurements (mean age 31 years), we observed significant gain in BMD T-score at the lumbar spine (0.58 ± 0.60, p = 0.003; mean gain of 0.62% areal BMD per year) and total femur T-score (0.24 ± 0.39, p = 0.041; mean gain of 0.25% areal BMD per year) after a mean follow-up period of 7.5 years. Compared to untreated subjects, a significant difference in evolution was demonstrated at the lumbar level (+0.58 ± 0.60 vs. -0.14 ± 0.42, p = 0.007). In untreated subjects with normal testosterone levels, a loss of BMD (-0.27 ± 0.37, p = 0.029; mean loss of 0.49% areal BMD per year) at the femoral neck was measured. This decline was equal to the predicted loss seen in the general male population. TRT results in BMD gain in patients with KS with testosterone deficiency, mainly at the lumbar spine. However, this effect is limited (0.62% per year). Patients who were not treated with TRT because of sufficient endogenous testosterone levels, showed only the predicted age-related bone loss during follow-up. The need for TRT in maintaining bone health in KS should be evaluated on an individual basis according to the degree of sex steroid deficiency.