An ultra-thin InAs layer with a thickness of 1 monolayer (ML) has been inserted at the direct (AlGaAs-on-GaAs) and inverted (GaAs-on-AlGaAs) interfaces in (001)-grown GaAs/AlGaAs quantum wells (QWs) with different well widths varying from 2 to 14nm, respectively. The in-plane optical anisotropy (IPOA) of these samples are investigated by reflectance difference spectroscopy. It is demonstrated that the IPOA of the QW sample with the ultra-thin InAs layer inserted at the direct interface is larger than that with InAs layer inserted at the inverted interface, which can be attributed to the smaller segregation effect of indium atoms at the direct interface compared to that at the inverted interface as confirmed by theoretical calculations based on 6 band k·p theory. Our results demonstrate that indium atoms present different segregation lengths at the direct and inverted interfaces of GaAs/AlGaAs QWs.