Exchange anisotropy occurs at the interface between an anti ferromagnetic (AF) layer and a ferromagnetic layer, and results in a ferromagnet hysteresis loop displaced along the field axis. We have performed off-specular neutron reflectometry in order to characterize the domain structure in Co layers that are exchange biased by FeMn. This allows us to determine the domain direction distributions and lateral magnetic correlation lengths for the Co layers as a function of field with the exchange bias in two different thermally prepared states: high field cooled and zero field cooled. We find that a reversing exchange-biased layer is characterized by a very short (submicron) magnetic lengthscale, indicating domains much smaller than those in the ferro-magnet after ac demagnetization at temperatures above the blocking temperature. This indicates that the bias is not spatially uniform across the entire interface, underlining the complexity of the AF spin structure.