There is broad agreement that neighborhood contexts are important for adolescent development, but there is less consensus about their association with adolescent smoking and alcohol use. Few studies have examined associations between neighborhood socioeconomic contexts and smoking and alcohol use while also accounting for differences in family and peer risk factors for substance use. Data drawn from the Seattle Social Development Project (N = 808), a gender-balanced (female = 49%), multiethnic, theory-driven longitudinal study originating in Seattle, WA, were used to estimate trajectories of smoking and alcohol use from 5th to 9th grade. Time-varying measures of neighborhood socioeconomic, family, and peer factors were associated with smoking and alcohol use at each wave after accounting for average growth in smoking and alcohol use over time and demographic differences. Results indicated that living in more socioeconomically disadvantaged neighborhoods, lower family income, lower family general functioning, more permissive family smoking environments, and affiliation with deviant peers were independently associated with increased smoking. Lower family functioning, more permissive family alcohol use environments, and deviant peers were independently associated with increased alcohol use. The effect of neighborhood disadvantage on smoking was mediated by family income and deviant peers while the effect of neighborhood disadvantage on alcohol use was mediated by deviant peers alone. Family functioning and family substance use did not mediate associations between neighborhood disadvantage and smoking or alcohol use. The results highlight the importance of neighborhood, family, and peer factors in early adolescent smoking and alcohol use. Future studies should examine the unique association of neighborhood disadvantage with adolescent smoking net of family socioeconomics, functioning, and substance use, as well aspeer affiliations. Better understanding of the role of contextual factors in early adolescent smoking and alcohol use can help bolster efforts to prevent both short and long harms from substance use.