Efficient behavior requires the rapid attentional selection of task-relevant objects. Preparatory activity of target-selective neurons in visual cortex is thought to support attentional selection, guiding spatial attention and favoring processing of target-matching input. However, naturalistic searches are often guided by nontargets, including target-associated “anchor” objects. For instance, when looking for a pen, we may direct our attention to the office desk on which we expect to find it. Here, using fMRI and eye tracking in a context-guided search task, we tested whether preparatory activity in visual cortex reflected the target, the guiding anchor object, or both. Participants learned associations between targets and anchors, reversing across two scene contexts, before searching for these targets. Participants’ first fixations were reliably guided by the associated anchor. Preparatory activity in lateral occipital cortex (LOC) and right intraparietal sulcus (IPS) represented the target-associated anchor rather than the target. These results shed light on the neural basis of context-guided search in structured environments.