Background: There is increasing evidence that the reliance on synthetic chemical pesticides and mineral fertilizers in agriculture has significant negative environmental and/or health impacts and poses a risk for future food security. Systematic reviews/meta-analyses showed that organic production systems, which omit the use of agrochemicals, produce crops with lower yields, but superior nutritional composition. However, the agronomic parameters responsible for differences in crop yields and nutritional quality are poorly understood. Methods: Here we report results for four field vegetable crops from the Nafferton Factorial Systems Comparison (NFSC) trial. This long-term factorial field experiment was designed to (i) identify effects of growing season/climatic variation, and contrasting rotational designs, crop protection protocols and fertilization regimes used in organic and conventional systems on crop health, yield and nutritional parameters and (ii) estimate the relative importance of climatic and agronomic drivers for crop health, yield and nutritionally relevant quality parameters. Quality parameters monitored in harvested products, included phenolic, glucosinolate, vitamin C, vitamin B9, carotenoid, cadmium (Cd), nickel (Ni), lead (Pb) and glycoalkaloid concentrations. Results: Climatic conditions during the growing season were found to have a larger impact on crop yield and quality than the agronomic factors (pre-crop, crop protection, fertilization) studied. However, the (i) interactions between growing season with contrasting climatic conditions and agronomic factors identified by ANOVA for crop health, yield and quality parameters and (ii) the associations between the three climatic drivers (precipitation, temperature, radiation) and crop yield and quality parameters differed substantially between the four crop plant species. Among the agronomic factors, fertilization had a substantially larger impact compared with both pre-crop and crop protection. Specifically, crop yields were found to be significantly increased by the use of (i) conventional fertilization and crop protection methods in potato, (ii) conventional fertilization, but organic crop protection methods in cabbage, and (iii) conventional fertilization regimes in lettuce, while none of the agronomic factors had a significant effect on onion yields. When important crop pest and diseases were assessed, (i) conventional crop protection resulted in significantly lower late blight severity in potato, while (ii) organic crop protection resulted in lower bird damage and cabbage root fly (CRF) incidence in cabbages, and Sclerotinia incidence in lettuce and (iii) organic fertilization resulted in lower CRF and Sclerotinia incidence in cabbage and lettuce respectively. When concentrations of nutritionally relevant phytochemicals were compared, organic fertilization resulted in significantly higher phenolic concentrations in potato, cabbage and lettuce, higher glucosinolate and carotenoid concentrations in cabbage, higher vitamin C concentrations in potato and cabbage and higher vitamin B9 concentrations in potato and lettuce—but lower concentrations of toxic glycoalkaloids in potato. Significant effects of crop protection protocols on phytochemical concentrations were only detected in cabbage with conventional crop protection resulting in higher glucosinolate and vitamin B9 concentrations. When toxic metal concentrations were compared, organic fertilization resulted in significantly lower Cd concentrations in all four crops and lower Ni concentrations in potato, cabbage and onion. Significant effects of crop protection were only detected in cabbage, where organic crop protection resulted in lower Ni concentrations. Pb concentrations were not affected by any of the agronomic factors. The potential implications of results for improving (i) strategies to reduce the use of non-renewable resources and environmental impacts of vegetable production and (ii) the productivity of organic and other low-input vegetable production systems without compromising food quality are discussed. Conclusions: The study confirms that organic vegetable production protocols result in higher concentrations of phenolics and other nutritionally desirable phytochemicals, but lower concentrations of the toxic metals Cd and Ni in harvested products. It also demonstrates, for the first time, that this is primarily due to differences in fertilization regimes. The finding that in three of the four crops (cabbage, lettuce and onion) the application of synthetic chemical crop protection products had no measurable positive impact on crop health and yield should be considered in the context of the growing concern about health impacts of pesticide use in field vegetable crops.