According to the stem cell hypothesis, breast carcinogenesis may be related to the breast stem cell pool size. However, little is known about associations of breast cancer risk factors, such as anthropometric measures, with the expression of stem cell markers in noncancerous breast tissue. The analysis included 414 women with biopsy-confirmed benign breast disease in the Nurses' Health Study and Nurses' Health Study II. Birthweight, weight at age 18, current weight, and current height were reported via self-administered questionnaires. IHC staining of stem cell markers (CD44, CD24, and aldehyde dehydrogenase family 1 member A1) in histopathologically normal epithelial and stromal breast tissue was quantified using an automated computational image analysis system. Linear regression was used to examine the associations of early-life and adult anthropometric measures with log-transformed stem cell marker expression, adjusting for potential confounders. Birthweight [≥10.0 vs. <5.5 lbs: β (95% confidence interval) = 4.29 (1.02, 7.56); P trend = 0.001 in the stroma] and adult height [≥67.0 vs. <63.0 inch: 0.86 (0.14, 1.58); P trend = 0.02 in the epithelium and stroma combined] were positively associated with CD44 expression. Childhood body fatness was inversely associated (P trend = 0.03) whereas adult height was positively associated with CD24 expression in combined stroma and epithelium (P trend = 0.03). Our data suggest that anthropometric measures, such as birthweight, adult height, and childhood body fatness, may be associated with the stem cell expression among women with benign breast disease. Anthropometric measures, such as birthweight, height, and childhood body fatness, may have long-term impacts on stem cell population in the breast.