Emerging literature emphasises the association between neighbourhood conditions and late life depression. Childhood experiences, crucial for life course development of mental health, may modify how neighbourhood affects subsequent depression. This study assessed the longitudinal associations of access to services and neighbourhood nuisance with depression among older adults, and tested whether these associations varied by exposure to childhood stressors. Data were drawn from the cross-national Survey of Health, Ageing and Retirement in Europe, a prospective cohort study between 2004/2005 and 2015, representative for European adults over the age of 50. Individual perceptions of neighbourhood were measured at baseline; childhood stressors, defined as socioeconomic conditions, adverse experiences and health problems, were collected retrospectively. Multilevel logistic regression estimated the risk of depression (n = 10,328). Access to services were negatively (OR = 0.78, 95% CI 0.68–0.90) and neighbourhood nuisance positively (OR = 1.36, 95% CI 1.18–1.56) associated with the probability of depression during follow-up. We found interactions between neighbourhood and childhood socioeconomic conditions, but not with adverse experiences and health problems. While older adults who grew up in better childhood socioeconomic conditions benefited more from living in a residential area with good access to services, they were at higher risk of developing depression when residing in areas with more neighbourhood nuisances. Older adults' mental health can benefit from better access to public transportation and neighbourhood amenities, while physical and social problems in the local area increase the risk of depression. Importantly, socioeconomic circumstances in early life may influence vulnerability to neighbourhood effects in older age.