Past reports indicated that total-body irradiation at low to moderate doses could be responsible for cardiovascular disease risks, but the mechanism remains unclear. The purpose of this study was to investigate the association between radiation exposure and atherosclerosis, an underlying pathology of cardiovascular diseases, in the Japanese atomic bomb survivors. We performed a cross-sectional study measuring 14 clinical-physiological atherosclerosis indicators during clinical exams from 2010 to 2014 in 3274 participants of the Adult Health Study cohort. Multivariable analyses were performed by using a structural equation model with latent factors representing underlying atherosclerotic pathologies: (1) arterial stiffness, (2) calcification, and (3) plaque as measured with indicators chosen a priori on the basis of clinical-physiological knowledge. Radiation was linearly associated with calcification (standardized coefficient per Gy 0.15, 95 % confidence interval: CI [0.070, 0.23]) and plaque (0.11, 95 % CI [0.029, 0.20]), small associations that were comparable to about 2 years of aging per Gy of radiation exposure, but not with arterial stiffness (0.036, 95 % CI [− 0.025, 0.095]). The model fitted better and had narrower confidence intervals than separate ordinary regression models explaining individual indicators independently. The associations were less evident when the dose range was restricted to a maximum of 2 or 1 Gy. By combining individual clinical-physiological indicators that are correlated because of common, underlying atherosclerotic pathologies, we found a small, but significant association of radiation with atherosclerosis.