Abstract Background Atherosclerosis is the main underlying cause of cardiovascular disease (CVD). Existing CVD risk assessment tools do not consider the burden of subclinical atherosclerosis. The presence of carotid plaques on carotid ultrasound is a well-known marker of subclinical atherosclerosis. The accumulation of population-scale data on the presence of atherosclerotic plaques, along with deep phenotyping, can allow not only to address the effectiveness of carotid ultrasound in routine clinical practice, but to shed light on the biology of atherosclerosis development. Purpose To develop an effective deep learning model for plaque detection in carotid ultrasound images in the UK Biobank. Methods We used 680 carotid ultrasound images with manually annotated plaques to train a deep learning model employing the YOLOv8 architecture. Different augmentation techniques were used to increase the generalizability of the model. The developed model was applied to automatically detect plaques in raw ultrasound images from 19,507 UK Biobank participants. Logistic and Cox regression were used to explore the associations of plaque presence and number as predicted by the model with conventional CVD risk factors and the risk of future CVD events over follow-up. To explore the genetic architecture of subclinical atherosclerosis, we conducted a genome-wide association study (GWAS) on plaque presence, followed by meta-analysis with data from the CHARGE Consortium. Results Our plaque detection model achieved high classification metrics of accuracy, sensitivity, and specificity (89.3%, 89.5%, and 89.2%, respectively) and detected atherosclerotic plaques in 44% of UK Biobank participants. As expected, plaques were more common among men than women and their prevalence increased linearly with age. Both plaque presence and number of plaques were correlated with conventional CVD risk factors including diabetes, hypertension, and hyperlipidemia, and showed strong associations with future risk of incident CVD events (Hazard Ratio for plaque presence: 1.48 [95%CI: 1.21-1.82], for 2 plaques or more: 1.65, [95% CI: 1.28-2.13]). Incorporating plaque-derived phenotypes minimally altered the C-index of the time-to-event model. GWAS meta-analysis of carotid plaque presence revealed 5 previously known loci, as well as a significant locus including the LPA gene that had not previously been associated with carotid plaque. Conclusion We have developed and implemented an efficient plaque detection model to data from the UK Biobank, which holds significant promise for studying atherosclerosis at a population-wide scale through integration with multiomics data and electronic health records.