ABSTRACT The formation of giant planets can be studied through direct imaging by observing planets both during and after formation. Giant planets are expected to form either by core accretion, which is typically associated with low initial entropy (cold-start models) or by gravitational instability, associated with high initial entropy of the gas (hot-start models). Thus, constraining the initial entropy can provide insight into a planet’s formation process and determines the resultant brightness evolution. In this study, we find that, by observing planets in nearby moving groups of known age both through direct imaging and astrometry with Gaia, it will be possible to constrain the initial entropy of giant planets. We simulate a set of planetary systems in stars in nearby moving groups identified by BANYAN Σ and assume a model for planet distribution consistent with radial-velocity detections. We find that Gaia should be able to detect approximately 25 per cent of planets in nearby moving groups greater than $\sim 0.3\, M_\text{J}$. Using 5σ contrast limits of current and future instruments, we calculate the flux uncertainty, and using models for the evolution of the planet brightness, we convert this to an initial entropy uncertainty. We find that future instruments such as METIS on E-ELT as well as GRAVITY and VIKiNG with VLTI should be able to constrain the entropy to within 0.5 kB/baryon, which implies that these instruments should be able to distinguish between hot- and cold-start models.