Age and depression may interact to produce a "double jeopardy" for cognitive impairment, and executive functioning, in cognitively unimpaired aging. Few studies have considered middle age or the ethnoracial diversity of subjects, despite evidence of more severe cognitive outcomes in historically minoritized people. In this pilot study, we investigated the impact of age on depression-related cognitive impairment and the underlying brain volumes in middle-aged non-Hispanic White adults (116), and Hispanic and Black adults (60), with a total number of 176 adults. The result shows a significant interaction between age and depression for executive functioning, specifically for middle-aged Hispanic and Black adults, but not non-Hispanic White adults. Prefrontal cortex volumes, which were reduced in the Black and Hispanic compared to the non-Hispanic White adults, partially mediated the relationship between depression level and executive functioning, across age and ethnoracial group. Collectively, these results suggest that the negative impact of depression on executive functioning and Prefrontal cortex volumes integrity may be exacerbated by age and that historically minoritized people may be particularly sensitive to this double jeopardy.