Children’s time investments in various activities may be important for reducing socioeconomic status (SES) gaps in educational and mental health outcomes. Using detailed time use diaries of Australian children aged 4–14, we observe that children from low SES backgrounds spend more time on digital media and less time on out-of-school enrichment activities, organised or for leisure. We explain the SES gap by employing a human capital development panel model and find that this difference contributes about 3% to the observed SES gap in numeracy skills. The results are supported by exogeneity tests and numerous robustness checks. The contribution is larger for males, older age groups, and if the cumulative effect on learning is considered. No clear results are found for literacy skills and mental health outcomes. The findings imply that interventions promoting a shift away from digital media towards out-of-school enrichment activities could help reduce the SES gap in human capital.