The Gaze Following Patch (GFP) is located in the posterior temporal cortex and has been described as a cortical module dedicated to processing other people's gaze direction in a domain-specific manner. Thus, it appears to be the neural correlate of Baron-Cohen's Eye-Direction Detector (EDD) which is one of the core modules in his Mindreading System - a neurocognitive model for the Theory of Mind concept. Inspired by Jerry Fodor's ideas on the modularity of the mind, Baron-Cohen proposed that, among other things, the individual modules are domain-specific. In the case of the EDD this means that it exclusively processes eye-like stimuli to extract gaze direction and that other stimuli, that may carry directional information as well, are processed elsewhere. If the GFP is indeed EDD's neural correlate it must meet this expectation. To test this, we compared the GFP's BOLD activity during gaze-direction following with the activity during arrow-direction following in the present human fMRI study. Contrary to the expectation based on the assumption of domain specificity we did not find a differentiation between gaze and arrow-direction following. In fact, we were not able to reproduce the GFP as presented in previous studies. A possible explanation is that in the present study - unlike previous work -, the gaze stimuli did not contain an obvious change of direction that represented a visual motion. Hence, the critical stimulus component responsible for the identification of the GFP in previous experiments might have been visual motion.Significance Statement This study presents evidence against the notion of domain-specificity of an area in the posterior temporal cortex (the gaze-following-patch; GFP) previously described to specifically serve eye gaze following. This conclusion is suggested by the finding that using arrows to identify a target object among distractors is accompanied by a comparable or even larger BOLD response than when the participants are asked to use the gaze direction of a demonstrator face for target selection. The fact that even the best candidate to date, the posterior temporal GFP, does not stand up to critical scrutiny casts doubt on the assumption that the brain uses a specific module to enable gaze following, as proposed by Simon Baron-Cohen.