Clinical risk scores for coronary artery disease (CAD) are used in clinical practice to select patients for diagnostic testing and therapy. Several studies have proposed that polygenic risk scores (PRSs) can improve the prediction of CAD, but the scores need to be validated in clinical populations with accurately characterized phenotypes. We assessed the predictive power of the three most promising PRSs for the prediction of coronary atherosclerosis and obstructive CAD. This study was conducted on 943 symptomatic patients with suspected CAD for whom the phenotype was accurately characterized using anatomic and functional imaging. Previously published genome-wide polygenic scores were generated to compare a genetic model based on PRSs with a model based on clinical data. The test and PRS cohorts were predominantly Caucasian of northern European ancestry. All three PRSs predicted coronary atherosclerosis and obstructive CAD statistically significantly. The predictive accuracy of the models combining clinical data and different PRSs varied between 0.778 and 0.805 in terms of the area under the receiver operating characteristic (AUROC), being close to the model including only clinical variables (AUROC 0.769). The difference between the clinical model and combined clinical + PRS model was not significant for PRS1 (p=0.627) and PRS3 (p=0.061). Only PRS2 slightly improved the predictive power of the model (p=0.04). The likelihood ratios showed the very weak diagnostic power of all PRSs. The addition of PRSs to conventional risk factors did not clinically significantly improve the predictive accuracy for either coronary atherosclerosis or obstructive CAD, showing that current PRSs are not justified for routine clinical use in CAD.