Lipid profiles are influenced by both noise and genetic variants. However, little is known about the associations of occupational noise and genetic variants with age-related changes in blood lipids, a crucial event in the initiation and evolution of atherosclerotic cardiovascular diseases. We aimed to evaluate the associations of blood lipid change rates with occupational noise and genetic variants in stress hormone biosynthesis-based genes. This cohort was established in 2012 and 2013 and was followed up until 2017. A total of 952 participants were included in the final analysis and all of them were categorized to two groups, the exposed group and control group, according to the exposed noise levels in their working area. Single nucleotide polymorphisms (SNPs) in stress hormone biosynthesis-based genes were genotyped. Five physical examinations were conducted from 2012 to 2017 and lipid measurements were repeated five times. The estimated annual changes (EACs) of blood lipid were calculated as the difference in blood lipid levels between any 2 adjacent examinations divided by their time interval (year). The generalized estimating equations for repeated measures analyses with exchangeable correlation structures were used to evaluate the influence of exposing to noise (versus being a control) and the SNPs mentioned above on the EACs of blood lipids. We found that the participants experienced accelerated age-related decline in high-density lipoprotein cholesterol (HDL-C) levels as they were exposed to noise (β = −0.38, 95% confidence interval (CI), −0.66 to −0.10, P = 0.007), after adjusting for work duration, gender, smoking, alcohol consumption, and pack-years. This trend was only found in participants with COMT-rs165815 TT genotype (β = −1.19, 95% CI, −1.80 to −0.58, P < 0.001), but not in those with the CC or CT genotypes. The interaction of noise exposure and rs165815 was marginally significant (Pinteraction = 0.010) after multiple adjustments. Compared with DDC-rs11978267 AA genotype carriers, participants carrying rs11978267 GG genotype had decreased EAC of triglycerides (TG) (β = −5.06, 95% CI, −9.07 to −1.05, P = 0.013). Participants carrying DBH-rs4740203 CC genotype had increased EAC of total cholesterol (TC) (β = 1.19, 95% CI, 0.06 to 2.33, P = 0.039). However, these findings were not statistically significant after multiple adjustments. These results indicated that Occupational noise exposure was associated with accelerated age-related decreases in HDL-C levels, and the COMT-rs165815 genotype appeared to modify the effect of noise exposure on HDL-C changes among the occupational population.