Most cancers occur more frequently in men. Numerous explanations for this excess risk have been proposed, yet no study has quantified the degree to which height explains the sex difference even though greater height has been associated with increased risk for many cancers. During the period from 2000 to 2002, 65308 volunteers aged 50 to 76 years were recruited to the Vitamins And Lifestyle (VITAL) study. Cancers of shared anatomic sites (n = 3466) were prospectively identified through 2009 through the Surveillance, Epidemiology, and End Results cancer registry. Age- and race-adjusted hazard ratios (HRs) for the associations between sex and incident cancers were estimated using Cox proportional hazards models, with and without adjustment for height and height squared as measures of body size. Men had a 55% increased risk of cancer at shared sites (HR = 1.55; 95% confidence interval [CI] = 1.45 to 1.66). When height was accounted for, 33.8% (95% CI = 10.2% to 57.3%) of the excess risk for men was explained by the height differences between sexes. The proportion mediated by height was 90.9%, 57.3%, and 49.6% for kidney, melanoma, and hematologic malignancies, respectively, with little evidence that height mediates the sex difference for gastrointestinal tract, lung, and bladder cancers. For comparison, more than 35 lifestyle and medical risk factors only explained 23.1% of the sex difference in cancer risk at shared sites. Height is an important explanatory factor for the excess risk for men for many shared-site cancers. This suggests that some of the excess risk is due to factors associated with height (eg, number of susceptible cells in a specific organ or growth-influencing exposures in childhood).