In this work, the influence of different original microstructure on the rolling contact fatigue (RCF) property of D2 wheel steel under oil lubrication condition is thoroughly studied by the GPM-30 rolling fatigue tester. Before pre-wear, the RCF life of the tempered sorbite (TS) sample and the lamellar pearlite + proeutectoid ferrite (P + F) sample are 7.2 × 105cycles and 5.4 × 105 cycle, respectively. After pre-wear, the RCF life of the TS sample is 4 times higher than that of the original TS sample. However, the RCF life of the P + F sample is 4 times lower than that of the original P + F sample. After pre-wear, the formation of fatigue wear cracks at the P + F sample surface causes the flaking of hardening layer and acts as a crack initiation site to accelerate the RCF failure. While the hardening layer of the TS sample is not flaked. The formation of hardening layer and the decrease of surface roughness of the TS sample improve the RCF life. During the RCF crack propagation process, the ferrite grains are refined obviously at the edge of cracks due to huge internal stress on the inner wall of the RCF cracks. The thickness of refined ferrite grains of the TS sample at the edge of RCF cracks is higher than that of the P + F sample.