Anticorrelation between the default and dorsal attention networks is a central feature of human functional brain organization. Hallmarks of aging include impaired default network modulation and declining medial temporal lobe (MTL) function. However, it remains unclear if this anticorrelation is preserved into older adulthood during task performance, or how this is related to the intrinsic architecture of the brain. We hypothesized that older adults would show reduced within- and increased between-network functional connectivity (FC) across the default and dorsal attention networks. To test this hypothesis, we examined the effects of aging on task-related and intrinsic FC using functional magnetic resonance imaging during an autobiographical planning task known to engage the default network and during rest, respectively, with young (n = 72) and older (n = 79) participants. The task-related FC analysis revealed reduced anticorrelation with aging. At rest, there was a robust double dissociation, with older adults showing a pattern of reduced within-network FC, but increased between-network FC, across both networks, relative to young adults. Moreover, older adults showed reduced intrinsic resting-state FC of the MTL with both networks suggesting a fractionation of the MTL memory system in healthy aging. These findings demonstrate age-related dedifferentiation among these competitive large-scale networks during both task and rest, consistent with the idea that age-related changes are associated with a breakdown in the intrinsic functional architecture within and among large-scale brain networks.