Childhood nature experiences have been identified as critical factors of Nature Relatedness (NR) at the individual level. This exploratory study delved into the distinct contribution of different forms of childhood experiences to NR in early adulthood, focusing on the living environment and actual contact with nature, along with nature education and the influence of people around. NR scores and self-reported experiences of Chinese college students (n = 431) were collected using a questionnaire survey. Objective environmental indicators (normalized difference vegetation index, land cover, and nighttime light) in 13 buffers of different sizes were adopted to precisely depict their historical living environments. Based on the results of bivariate correlations and hierarchical linear regressions, the quantity of natural components in the childhood living environment did not predict long-term NR. Nighttime light and impervious surface at large geographic scales had significant but weak negative correlations with NR. Experiences of actual contact with nature, especially the experiences of interactive behaviors rather than merely visiting natural spaces, are better predictors than the objective environment. The interactive behaviors of observing natural processes and cultivating plants had significantly independent contributions to NR, even when other factors were included in the regression models. Nature education could not replace the role of actual contact with nature. This study implies that an urbanized living environment will not necessarily reduce the level of NR of future generations. Rather than simply increasing the quantity of natural components in urban areas, it is more important to support children’s interactive behaviors with nature.