Empathy is at the core of our social world, yet multidomain factors that affect its development in socially sensitive periods, such as adolescence, are incompletely understood. To address this gap, this study investigated associations between social, environmental and mental health factors, and their temporal changes, on adolescent empathetic behaviors/emotions and, for comparison, callous unemotional (CU) traits and behaviors, in the early longitudinal Adolescent Brain Cognitive Development sample (baseline: n = 11062; 2-year follow-up: n = 9832, median age = 119 and 144 months, respectively). Caregiver affection towards the youth, liking school, having a close friend, and importance of religious beliefs/spirituality in the youth's life were consistently positively correlated with empathetic behaviors/emotions across assessments (p<0.001, Cohen's f = ~0.10). Positive family dynamics and cohesion, living in a neighborhood that shared the family's values, but also parent history of substance use and (aggregated) internalizing problems were additionally positively associated with one or more empathetic behaviors at follow-up (p<0.001, f = ~0.10). In contrast, externalizing problems, anxiety, depression, fear of social situations, and being withdrawn were negatively associated with empathetic behaviors and positively associated with CU traits and behaviors (p<0.001, f = ~0.1-0.44). The latter were also correlated with being cyberbullied and/or discriminated against, anhedonia, and impulsivity, and their interactions with externalizing and internalizing issues. Significant positive temporal correlations of behaviors at the two assessments indicated positive (early) developmental empathetic behavior trajectories, and negative CU traits' trajectories. Negative changes in mental health adversely moderated positive trajectories and facilitated negative ones. These findings highlight that adolescent empathetic behaviors/emotions are positively related to multidomain protective social environmental factors, but simultaneously adversely associated with risk factors in the same domains, as well as bully victimization, discrimination, and mental health problems. Risk factors instead facilitate the development of CU traits and behaviors.