The novel Gaia multi peak (GMP) technique has proven to be able to successfully select dual and lensed active galactic nuclei (AGN) candidates at sub-arcsecond separations. Both populations are important because dual AGN represent one of the central, still largely untested, predictions of ΛCDM cosmology, and compact lensed AGN allow us to probe the central regions of the lensing galaxies. In this work, we present high-spatial-resolution spectroscopy of 12 GMP-selected systems. We used the adaptive-optics assisted integral-field spectrograph MUSE at the VLT to resolve each system and investigate the nature of each component. All targets show the presence of two components confirming the GMP selection. We classify 4 targets as dual AGN, 3 as lensed quasar candidates, and 5 as a chance alignment of a star and an AGN. With separations ranging from 0.30″ to 0.86″, these dual and lensed systems are among the most compact systems discovered to date at z > 0.5. This is the largest sample of distant dual AGN with sub-arcsecond separations ever presented in a single paper.