Urban allotment gardens (UAGs) are expanding worldwide, especially in large cities. Environmental pressures (direct and diffuse pollution, gardener practice, geogenic contamination) often result in the accumulation of potentially harmful trace elements in garden soils. The objectives of this study were to assess the spatial variability of trace element distribution in UAGs from city, garden, and plot scale in four European cities; to provide a baseline understanding and identify abnormal values under environmental pressures; and to evaluate the potential of portable X-ray fluorescence screening as a useful tool in soil management. The four cities (Ayr and Greenock (Scotland), Lisbon (Portugal), Nantes (France)) provided a wide range of environmental pressures on soils. The locations of the 14 allotment gardens were identified in consultation with the local municipality in each city to reflect various land uses or according to previous evaluation of soil quality. Soil sampling was carried out in 66 plots in total, from which 3 datasets were produced: (i) basic soil properties and trace element concentrations from a composite sample of topsoil for each plot (trace elements quantified by inductively coupled plasma–optical emission spectrometry/mass spectrometry (ICP-OES/MS) or using in-lab portable X-ray fluorescence (PXRF); (ii) in situ PXRF measurement on composite samples (263 plots in Nantes); and (iii) composite samples from 32 small areas within 4 plots in one garden of Nantes. The results were analyzed to assess the spatial variability of soil properties. At city and garden scale, the variability observed for basic soil properties and major elements is dominated by local geology/parent material (pH, CaCO3, Fe) and gardening practice (OM, CaCO3), which vary between each country. The range of trace element concentrations is similar between each city except for Greenock. Extreme values are observed for Cu, Pb, and Zn reflecting human disruption. In most situations, the trace element contamination was explained through the historical and environmental situations of the site. The PXRF screening method proved useful in providing detailed mapping for hot spot detection or delineation, providing support for soil management at plot and garden scale. As anticipated, basic soil properties appear to be controlled by the parent material. At plot and garden scale, the trace element variability shows the influence of land use history and background and strong inputs from external factors (e.g., by industrial activity or traffic emission). The PXRF screening method appears to be an efficient solution for soil management as it can be used to discriminate zones which may require restriction on cultivation.