Timely availability of weed infestation maps is a key prerequisite for the implementation of site-specific weed management practices. Low-altitude aerial imagery obtained from unmanned aerial vehicles (UAVs) has shown significant potential for weed detection in crops. However, most studies focused on wide-spaced row crops such as maize and sunflower and evaluated proposed methods on single or few well-characterized experimental sites at specific points in time representing a limited range of application scenarios. This study evaluated the feasibility of weed detection in on-farm wheat fields characterized by a narrow row spacing, throughout the early and late developmental stages using UAV imagery and ground-based high-resolution imagery. Image data was obtained for nine sites, representing a wide range of management and pedoclimatic conditions. These sites can be seen as a representative sample of scenarios that would be encountered in practice. A high within- and across-site as well as temporal variation was observed for weed infestation levels and weed population species composition, highlighting the need for spatially and temporally resolved weed mapping. Image-based classification of vegetation objects as crop or weed plants was achieved with an accuracy of 0.88 and 0.72 in ground-based high-resolution images and UAV-based aerial images captured from an altitude of 10 m, respectively. The accuracy of pixel-wise, vegetation-index-based weed infestation estimation during the late vegetative stages varied strongly across sites. Our results highlight the critical importance of a high ground resolution for weed detection using object-based image analysis during the critical growth stages of wheat and of robust methods that are applicable across a range of scenarios. This suggested that future research aiming at a rapid implementation of site-specific weed management in wheat should focus on the development of ground-based systems. Yet, aerial monitoring of wheat stands during late developmental stages using currently available equipment offers significant potential for reducing weed pressure with site-specific weed control measures in the context of crop rotations.