Heusler alloys are a material class exhibiting various magnetic properties, including antiferromagnetism. A typical application of antiferromagnets is exchange bias that is a shift of the magnetization curve observed in a layered structure consisting of antiferromagnetic and ferromagnetic films. In this study, a layered sample consisting of a Heusler alloy, MnVAl and a ferromagnet, Fe, is selected as a material system exhibiting exchange bias. Although the fully ordered MnVAl is known as a ferrimagnet, with an optimum fabrication condition for the MnVAl layer, the MnVAl/Fe layered structure exhibits exchange bias. The appearance of the antiferromagnetic property in the MnVAl is remarkable; however, the details have been unclear. To clarify the microscopic aspects on the crystal structures and magnetic moments around the MnVAl/Fe interface, cross-sectional scanning transmission electron microscope (STEM) observation, and synchrotron soft X-ray magnetic circular dichroism (XMCD) measurements were employed. The high-angle annular dark-field STEM images demonstrated clusters of MnVAl with the L2 phase distributed only around the interface to the Fe layer in the sample showing the exchange bias. Furthermore, antiferromagnetic coupling between the Mn- and Fe-moments were observed in element-specific hysteresis loops measured using the XMCD. The locally ordered L2 phase and antiferromagnetic Mn-moments in the MnVAl were suggested as important factors for the exchange bias.