What we have attended to in the past, as well as the stimulus context associated with past motor responses, have a strong impact on our current behavior. These influences have been investigated through inter-trial priming effects in visual search and sequence effects in action control, respectively. These two research fields are strongly complementary at the theoretical level and show striking similarities in their experimental-task structure, analyses, and results. Yet, they have developed largely separately. Here, we claim that such fragmentation impedes progress in these two research strands and highlight the potential benefits of intensifying crosstalk between visual search and action control in future research by exploiting the existing structural similarities with regard to sequence effects. We first discuss the main phenomena and theoretical explanations in each field, while emphasizing the similarities and differences between them. Then, we illustrate how the two fields could integrate each other’s insights—namely, how visual-search research could draw on the action-control literature to clarify the role of retrieval in selection and how action-control research could draw on the visual-search literature to explain response-related processes in more complex environments. We argue that combining the two research traditions is necessary for a coherent account of search-for-action behavior.