Background: Coronary artery calcium (CAC) is a marker of subclinical atherosclerosis and is a complex heritable trait with both genetic and environmental risk factors, including sex and smoking. Methods: We performed genome-wide association (GWA) analyses for CAC among all participants and stratified by sex in the COPDGene study (n = 6144 participants of European ancestry and n = 2589 participants of African ancestry) with replication in the Diabetes Heart Study (DHS). We adjusted for age, sex, current smoking status, BMI, diabetes, self-reported high blood pressure, self-reported high cholesterol, and genetic ancestry (as summarized by principal components computed within each racial group). For the significant signals from the GWA analyses, we examined the single nucleotide polymorphism (SNP) by sex interactions, stratified by smoking status (current vs. former), and tested for a SNP by smoking status interaction on CAC. Results: We identified genome-wide significant associations for CAC in the chromosome 9p21 region [CDKN2B-AS1] among all COPDGene participants (p = 7.1 × 10-14) and among males (p = 1.0 × 10-9), but the signal was not genome-wide significant among females (p = 6.4 × 10-6). For the sex stratified GWA analyses among females, the chromosome 6p24 region [PHACTR1] had a genome-wide significant association (p = 4.4 × 10-8) with CAC, but this signal was not genome-wide significant among all COPDGene participants (p = 1.7 × 10-7) or males (p = 0.03). There was a significant interaction for the SNP rs9349379 in PHACTR1 with sex (p = 0.02), but the interaction was not significant for the SNP rs10757272 in CDKN2B-AS1 with sex (p = 0.21). In addition, PHACTR1 had a stronger association with CAC among current smokers (p = 6.2 × 10-7) than former smokers (p = 7.5 × 10-3) and the SNP by smoking status interaction was marginally significant (p = 0.03). CDKN2B-AS1 had a strong association with CAC among both former (p = 7.7 × 10-8) and current smokers (p = 1.7 × 10-7) and the SNP by smoking status interaction was not significant (p = 0.40). Conclusions: Among current and former smokers of European ancestry in the COPDGene study, we identified a genome-wide significant association in the chromosome 6p24 region [PHACTR1] with CAC among females, but not among males. This region had a significant SNP by sex and SNP by smoking interaction on CAC.