We examined the associations between carbohydrate substitutions (total; low-, medium-, high-glycemic index (GI) carbohydrates) for fat or protein and risk of type 2 diabetes. The cohort comprised 25,943 male smokers among whom 1098 diabetes cases were identified from a national register during a 12-year follow-up. Diet was assessed by a validated food frequency questionnaire. The relative risks (RR) and confidence intervals (CI) for diabetes were analyzed using Cox proportional hazard modeling, and multivariate nutrient density models were applied to examine the associations between substitutions of macronutrients and diabetes risk. The risk of diabetes was lower when fat or protein was replaced with an isoenergetic amount (2% of energy intake) of carbohydrates, the multivariate RRs were 0.96 (95% CI: 0.94, 0.99) and 0.85 (95% CI: 0.80, 0.90), respectively. The lower risks were due to replacing saturated plus trans fatty acids, and meat, milk or plant protein with carbohydrates, respectively. Low-, medium- or high-GI carbohydrates did not associate with lower diabetes risk when replacing fat or fatty acids, except when total fat was replaced with medium-GI carbohydrates. Low-, medium- and high-GI carbohydrates had similar inverse associations with diabetes risk when they replaced total, meat or milk protein. Higher carbohydrate intake at the expense of fat, attributable to trans and saturated fatty acids, or protein was associated with decreased diabetes risk. Replacing fat or protein with lower-GI carbohydrates was not more beneficial than replacing it with higher-GI carbohydrates.