Cardiovascular disease (CVD) is a leading cause of morbidity and mortality worldwide. Dioxins can cause cardiovascular toxicity in experimental animals. The potential role of dioxin exposure as a preventable risk factor has attracted the attention of public health services, especially because dioxin exposure is a ubiquitous problem. We aimed to investigate and clarify the effect on CVD risk of moderate-to-high exposure to dioxins. This cross-sectional study investigated 914 residents without CVD near a deserted pentachlorophenol factory. CVD-related factors were measured to examine their associations with serum dioxin. We also investigated associations between serum dioxins and the Framingham risk score. Serum PCDD/F levels were significantly positively associated with CVD risk in both genders (Men: b = 0.023, P < 0.001; Women: b = 0.005, P < 0.001; All: b = 0.013, P < 0.001). After adjusting for confounding factors, participants with higher serum PCDD/F levels had a higher risk for CVD than did the reference group (serum PCDD/levels < 9.8 pg WHO 98-TEQ DF/g lipid) (25th to <50th percentile, adjusted odds ratio (AOR) = 2.96 [95% confidence interval (CI) = 1.13–7.75]; 50th to < 75th percentile, AOR = 3.37 [1.32–8.59]; ≥75th percentile, AOR = 6.22 [2.47–15.63]). We hypothesize that accumulated dioxins heightens the cardiovascular risk.