Weeds are a major constraint for crop production and food security. Chemical management, the most utilized method for weed control, has serious drawbacks. In this context, the development of more sustainable methods like site-specific weed management (SSWM) is highly deemed. With this study, we assessed the possibility of applying two convolutional neural networks (CNN) for the recognition of different classes of weeds in a winter wheat (T. aestivum) cultivation based on RGB images. By using this methodology, we were able to recognize with a high average precision (AP > 0.6) some species whose abundance and distribution in the field was correlated with the final wheat yield. We demonstrated that where the presence of R. raphanistrum, A. arvensis and P. rhoeas was high at the tillering stage and weed biodiversity indexes were low (Menhinich's Index, Simpson's Reciprocal Index and Shannon's Index), wheat yield was significantly reduced. In contrast, higher weed biodiversity mitigated yield losses. Therefore, CNN is a useful tool for early evaluation of the impact that weeds may have on yield, and it can be used as SSWM classifier for an early mapping of weeds, which is critical to improve our understanding of weed ecology dynamics in agricultural fields.