To test the idea that the slowing of simple information processing contributes to more general cognitive ageing, it is necessary to demonstrate that changes in the two variables are correlated as people grow older. Here, we examine the association between inspection time—a psychophysical measure of visual information processing—and general cognitive ability and the cognitive domains of visuospatial reasoning, processing speed, memory, and crystallised ability across five waves of testing in a 12-year period. The participants were members of the Lothian Birth Cohort 1936; there was a maximum of 1090 people with cognitive data at age 70 (Wave 1) and 426 at age 82 (Wave 5). At each testing wave the participants took the same 12 cognitive tests. Latent growth curve modelling in a structural equation modelling framework was used to examine the associations between intercepts and slopes of inspection time and other cognitive capabilities. Age-related changes (slope) in inspection time correlated 0.898 (p < 0.001) with changes (slope) in general cognitive ability over the 12 years. Inspection time changes correlated with changes in each of the four cognitive domains, but these associations were reduced to non-significance once the domains' loadings on general cognitive ability were taken into account (with the possible exception of memory, whose changes still had a marginal additional association with inspection time changes; β = 0.199, p = 0.030). The results are compatible with the idea that age-related slowing of processing speed contributes causally to the age-related declines in complex cognitive capability, but this is not the only interpretation of the present findings.