Single impurities in insulators are now often used for quantum sensors and single photon sources, while nanoscale semiconductor doping features are being constructed for electrical contacts in quantum technology devices, implying that new methods for sensitive, non-destructive imaging of single- or few-atom structures are needed. X-ray fluorescence (XRF) can provide nanoscale imaging with chemical specificity, and features comprising as few as 100000 atoms have been detected without any need for specialized or destructive sample preparation. Presently, the ultimate limits of sensitivity of XRF are unknown - here, gallium dopants in silicon are investigated using a high brilliance, synchrotron source collimated to a small spot. It is demonstrated that with a single-pixel integration time of 1 s, the sensitivity is sufficient to identify a single isolated feature of only 3000Ga impurities (a mass of just 350 zg). With increased integration (25 s), 650 impurities can be detected. The results are quantified using a calibration sample consisting of precisely controlled numbers of implanted atoms in nanometer-sized structures. The results show that such features can now be mapped quantitatively when calibration samples are used, and suggest that, in the near future, planned upgrades to XRF facilities might achieve single-atom sensitivity.