The large convergence angle of high numerical aperture (NA) objective lens is an obstacle to develop a high spatial resolution micro-reflectance difference spectrometer (micro-RDS). Here, we figure out the anisotropic signal mapping of the high NA objective lens induced to micro-RDS using mathematical approaches of vectorial ray-tracing method and Debye–Wolf integral. The spatial symmetry of the RD signal promises a value of zero by the integral over the light beam profile under the condition that the micro-RDS system is perfectly symmetrical. Thus, the RDS system needs to be more strictly symmetrical when a larger NA objective lens is utilized. For demonstration, a micro-RDS based on liquid crystal variable retarder was built. An in-situ calibration method was developed to eliminate the testing errors introduced by the asymmetry of the optical system. As a result, the measurement precision of the setup is better than ±0.002 when a 100× objective (NA=0.9) is used. Finally, to demonstrate the performance of the micro-RDS, we measured the layered black phosphorus using 50× and 100× objective lenses, respectively, and observed a scaled factor tuning the amplitude of the real part of RD signal when a sample is smaller than the light spot. The origins of the scaled factor are discussed.