This study sought to evaluate the associations between changes in glycemic status and changes in total body (TB), trunk, and appendicular fat (FM) and lean mass (LM) in men. A population-based study of men aged 20–66 years at baseline were included in cross-sectional (n = 430) and three-year longitudinal (n = 411) analyses. Prediabetes was defined as fasting glucose 100–125 mg/dL. Type 2 diabetes (T2D) was determined by: self-reported diabetes, current anti-diabetic drug use (insulin/oral hypoglycemic agents), fasting glucose (≥126 mg/dL), or non-fasting glucose (≥200 mg/dL). Body composition was evaluated by dual-energy X-ray absorptiometry. Longitudinal analyses showed that changes in TB FM and LM, and appendicular LM differed among glycemic groups. Normoglycemic men who converted to prediabetes lost more TB and appendicular LM than men who remained normoglycemic (all, p < 0.05). Normoglycemic or prediabetic men who developed T2D had a greater loss of TB and appendicular LM than men who remained normoglycemic (both, p < 0.05). T2D men had greater gains in TB FM and greater losses in TB and appendicular LM than men who remained normoglycemic (all, p < 0.05). Dysglycemia is associated with adverse changes in TB and appendicular LM.