Radiation therapy for breast cancer is associated with increased risk of a second primary contralateral breast cancer, but the genetic factors modifying this association are not well understood. To determine whether a genetic risk score comprising single nucleotide polymorphisms in the nonhomologous end-joining DNA repair pathway is associated with radiation-associated contralateral breast cancer. This case-control study included a case group of women with contralateral breast cancer that was diagnosed at least 1 year after a first primary breast cancer who were individually matched to a control group of women with unilateral breast cancer. Inclusion criteria were receiving a first invasive breast cancer diagnosis prior to age 55 years between 1985 and 2008. Women were recruited through 8 population-based cancer registries in the United States, Canada, and Denmark as part of the Women's Environment, Cancer, and Radiation Epidemiology Studies I (November 2000 to August 2004) and II (March 2010 to December 2012). Data analysis was conducted from July 2017 to August 2019. Stray radiation dose to the contralateral breast during radiation therapy for the first breast cancer. A novel genetic risk score comprised of genetic variants in the nonhomologous end-joining DNA repair pathway was considered the potential effect modifier, dichotomized as high risk if the score was above the median of 74 and low risk if the score was at or below the median. The main outcome was risk of contralateral breast cancer associated with stray radiation dose stratified by genetic risk score, age, and latency. A total of 5953 women were approached for study participation, and 3732 women (62.7%) agreed to participate. The median (range) age at first diagnosis was 46 (23-54) years. After 5 years of latency or more, among women who received the first diagnosis when they were younger than 40 years, exposure to 1.0 Gy (to convert to rad, multiply by 100) or more of stray radiation was associated with a 2-fold increased risk of contralateral breast cancer compared with women who were not exposed (rate ratio, 2.0 [95% CI, 1.1-3.6]). The risk was higher among women with a genetic risk score above the median (rate ratio, 3.0 [95% CI, 1.1-8.1]), and there was no association among women with a genetic risk score below the median (rate ratio, 1.3 [95% CI, 0.5-3.7]). Among younger women with a high genetic risk score, the attributable increased risk for contralateral breast cancer associated with stray radiation dose was 28%. This study found an increased risk of contralateral breast cancer that was attributable to stray radiation exposure among women with a high genetic risk score and who received a first breast cancer diagnosis when they were younger than 40 years after 5 years or more of latency. This genetic risk score may help guide treatment and surveillance for women with breast cancer.