We examined the associations between serum cholesterol measures, statin use, and cognitive function measured in childhood and in old age. The possibility that lifelong (trait) cognitive ability accounts for any cross-sectional associations between cholesterol and cognitive performance in older age, seen in observational studies, has not been tested to date. Participants were 1,043 men and women from the Lothian Birth Cohort 1936 Study, most of whom had participated in a nationwide IQ-type test in childhood (Scottish Mental Survey of 1947), and were followed up at about age 70 years. Serum cholesterol measures included total cholesterol (TC), high-density lipoprotein cholesterol (HDL-C), triglycerides, and cholesterol:HDL cholesterol ratio. Cognitive outcome measures were age 70 IQ (using the same test as at age 11 years), general cognitive ability (g), processing speed, memory, and verbal ability. Higher TC, higher HDL-C, and lower triglycerides were associated with higher age 70 cognitive scores in most cognitive domains. These relationships were no longer significant after covarying for childhood IQ, with the exception a markedly attenuated association between TC and processing speed, and triglycerides and age 70 IQ. In the fully adjusted model, all conventionally significant (p < 0.05) effects were removed. Childhood IQ predicted statin use in old age. Statin users had lower g, processing speed, and verbal ability scores at age 70 years after covarying for childhood IQ, but significance was lost after adjusting for TC levels. These results suggest that serum cholesterol and cognitive function are associated in older age via the lifelong stable trait of intelligence. Potential mechanisms, including lifestyle factors, are discussed.