Context. Many protoplanetary discs are self-gravitating early in their lives. If they fragment under their own gravity, they form bound gaseous clumps that can evolve to become giant planets. Today, the fraction of discs that undergo fragmentation, and therefore also the frequency of conditions that may lead to giant planet formation via gravitational instability, is still unknown. Aims. We study the formation and evolution of a large number of star-disc systems, focusing on the early sizes of the discs and their likelihood to fragment. We investigate how the fraction of discs that fragments depends on the disc-size distribution at early times. Methods. We performed a population synthesis of discs from formation to dispersal. Whilst varying the infall radius, we study the relationship between early disc size and fragmentation. Furthermore, we investigate how stellar accretion heating affects the fragmentation fraction. Results. We find that discs fragment only if they become sufficiently large early in their lives. This size depends sensitively on where mass is added to the discs during the collapse of their parent molecular cloud core. Infall locations derived from pure hydrodynamic and non-ideal magnetised collapse simulations lead to large and small discs, respectively, and 22 and 0% fragmentation fractions, respectively, in populations representative of the initial mass function; however, the resulting synthetic disc size distribution is larger and smaller, respectively, than the observed Class 0 disc size distribution. By choosing intermediate infall locations, leading to a synthetic disc size distribution that is in agreement with the observed one, we find a fragmentation fraction of between 0.1 and 11%, depending on the efficiency of stellar accretion heating of the discs. Conclusions. We conclude that the frequency of fragmentation is strongly affected by the early formation process of the disc and its interaction with the star. The early disc size is mainly determined by the infall location during the collapse of the molecular cloud core and controls the population-wide frequency of fragmentation. Stellar accretion heating also plays an important role in fragmentation and must be studied further. Our work is an observationally informed step towards a prediction of the frequency of giant planet formation by gravitational instability. Upcoming observations and theoretical studies will further our understanding of the formation and early evolution of discs in the near future. This will eventually allow us to understand how infall, disc morphology, giant planet formation via gravitational instability, and the observed extrasolar planet population are linked.