National Child Development Study (NCDS) data are used to examine the negative impact of time out of paid employment on numeracy, as measured by a maths test at 16 and a functional numeracy test at 37. Restricting the sample to respondents who left full‐time education at 16 and accounting for maths at 16, we found negative correlations between time out of paid employment and adult numeracy scores. Using the whole sample, adult numeracy scores were regressed on maths at 16, family background and adult experiences. The longer the absence from paid employment, the greater the negative impact on adult numeracy. The relationship was strongest for men with poor maths at 16. This suggested that a certain level of maths was needed before skills were retained and not weakened by absence from paid employment. Training offered some protection against skill loss, as did women’s more diverse roles at home and work.