Female adolescents are less active than male peers in certain contexts including the neighborhood. Adolescents' physical activity can be explained by interactions between environmental and psychosocial factors, but few studies have tested such interactions in relation to context-specific behaviors. This study tested interactions between neighborhood environmental and psychosocial factors in relation to adolescents' context-specific physical activity. Data were collected in 2009–11 from 910 adolescents and a parent/guardian residing in the Baltimore/Seattle regions. Measures included adolescent-reported neighborhood leisure-time physical activity (LTPA) and non-neighborhood LTPA, accelerometer-based non-school moderate-to vigorous-physical activity (MVPA), psychosocial factors, and objective and parent-perceived neighborhood environmental factors. Gender-stratified mixed effects linear models tested associations of 6 environmental and 4 psychosocial factors and their interactions in relation to each physical activity outcome. The psychosocial factors had consistent associations with the physical activity outcomes but the environmental correlates were context-specific. Decisional balance (weighing of pros and cons of physical activity) moderated the association between recreation facility density and neighborhood LTPA among females, with a negative association only among those with high decisional balance (pros outweighed cons). Decisional balance also moderated associations of neighborhood walkability with non-school MVPA among females and non-neighborhood LTPA among males, with positive associations only among those with high decisional balance. Results support context-specific ecological models of physical activity. Targeting environmental factors that may promote opportunities for physical activity in specific contexts as well as adolescent decision-making may help promote their physical activity in those contexts, potentially leading to increased overall physical activity.