People tend to judge more recent events, relative to earlier ones, as the cause of some particular outcome. For instance, people are more inclined to judge that the last basket, rather than the first, caused the team to win the basketball game. This recency effect, however, reverses in cases of overdetermination: people judge that earlier events, rather than more recent ones, caused the outcome when the event is individually sufficient but not individually necessary for the outcome. In five experiments (N = 5507), we find evidence for the recency effect and the primacy effect for causal judgment. Traditionally, these effects have been a problem for counterfactual views of causal judgment. However, we argue that an extension of a recent counterfactual model of causal judgment explains both the recency and the primacy effect. In line with the predictions of our extended counterfactual model, we also find that, regardless of causal structure, people tend to imagine the counterfactual alternative to the more recent event rather than to the earlier one. Moreover, manipulating this tendency affects causal judgments in the ways predicted by this extended model: asking participants to imagine the counterfactual alternative to the earlier event weakens the interaction between recency and causal structure, and asking participants to imagine the counterfactual alternative to the more recent event strengthens the interaction between recency and causal structure. We discuss these results in relation to work on counterfactual thinking, causal modeling, and late-preemption.