Engaging cognitive control is essential to flexibly adapt to constantly changing environments. However, relatively little is known about how prior task experience impacts on the engagement of cognitive control in novel task environments. We aimed to clarify how individuals learn and transfer the engagement of cognitive control with a focus on the hierarchical and temporal aspects of task knowledge. Highlighting two distinct cognitive control processes, the engagement of cognitive control in advance (proactive control) and in response to conflicts (reactive control), we conducted six preregistered online experiments with both adults (Experiment 1, 3, and 5: N = 71, N = 108, and N = 70) and 9- to 10-year-olds (Experiment 2, 4, 6: N = 69, N = 108, and N = 70). Using two different experimental paradigms, we demonstrated that prior task experience of engaging reactive control makes adults and 9-to 10-year-olds respond in a reactive way in a subsequent similar-structured condition with different stimuli in which proactive control could have been engaged. This indicates that individuals do learn knowledge about the temporal structure of task goal activation and, on occasion, negatively transfer this knowledge. Furthermore, individuals exhibited these negative transfer effects in a similar-structured condition with different task goals and stimuli, indicating that they learn hierarchically-structured task knowledge. The collective findings suggest a new way of understanding how hierarchical and temporal task knowledge influences the engagement of cognitive control and highlight potential mechanisms underlying the near transfer effects observed in cognitive control training.