ABSTRACT Low-surface-brightness galaxies (LSBGs) – defined as systems that are fainter than the surface-brightness limits of past wide-area surveys – form the overwhelming majority of galaxies in the dwarf regime (M⋆ < 109 M⊙). Using NewHorizon, a high-resolution cosmological simulation, we study the origin of LSBGs and explain why LSBGs at similar stellar mass show the large observed spread in surface brightness. NewHorizon galaxies populate a well-defined locus in the surface brightness–stellar mass plane, with a spread of ∼3 mag arcsec−2, in agreement with deep Sloan Digital Sky Survey (SDSS) Stripe 82 data. Galaxies with fainter surface brightnesses today are born in regions of higher dark matter density. This results in faster gas accretion and more intense star formation at early epochs. The stronger resultant supernova feedback flattens gas profiles at a faster rate, which, in turn, creates shallower stellar profiles (i.e. more diffuse systems) more rapidly. As star formation declines towards late epochs ( z < 1), the larger tidal perturbations and ram pressure experienced by these systems (due to their denser local environments) accelerate the divergence in surface brightness, by increasing their effective radii and reducing star formation, respectively. A small minority of dwarfs depart from the main locus towards high surface brightnesses, making them detectable in past wide surveys (e.g. standard-depth SDSS images). These systems have anomalously high star formation rates, triggered by recent fly-by or merger-driven starbursts. We note that objects considered extreme or anomalous at the depth of current data sets, e.g. ‘ultra-diffuse galaxies’, actually dominate the predicted dwarf population and will be routinely visible in future surveys like the Legacy Survey of Space and Time (LSST).