Memory is the fundamental form of temporal complexity: when present but uncontrollable, it manifests as non-Markovian noise; conversely, if controllable, memory can be a powerful resource for information processing. Memory effects arise from/are transmitted via interactions between a system and its environment; as such, they can be either classical or quantum. From a practical standpoint, quantum processes with classical memory promise near-term applicability: they are more powerful than their memoryless counterpart, yet at the same time can be controlled over significant timeframes without being spoiled by decoherence. However, despite practical and foundational value, apart from simple two-time scenarios, the distinction between quantum and classical memory remains unexplored. Here, we analyse multi-time quantum processes with memory mechanisms that transmit only classical information forward in time. Complementing this analysis, we also study two related – but simpler to characterise – sets of processes that could also be considered to have classical memory from a structural perspective, and demonstrate that these lead to remarkably distinct phenomena in the multi-time setting. Subsequently, we systematically stratify the full hierarchy of memory effects in quantum mechanics, many levels of which collapse in the two-time setting, making our results genuinely multi-time phenomena.