Abstract Long reach Passive optical network (LR-PON) is an attractive solution to fulfill the ever-increasing bandwidth requirements due to propelling internet applications and competent to serve distant optical network units (ONUs). Wavelength division multiplexed (WDM) PON systems experience distance and performance limiting constraint termed as Dispersion. In order to compensate dispersion effects, Fiber bragg gratings (FBGs) and Dispersion compensation fibers (DCFs) are incorporated extensively in PONs. Performance of DCF is better than FBG in terms of dispersion compensation, but it comes at the cost of 3 $/m (very expensive). Therefore, long reach ultra dense WDM-PON systems are needed with incorporation of economical and high performance DCMs. Three newly constructed hybrid DCMs are investigated such as FBG-DCF (module 1), OPC-DCF (module 2), and FBG-DCF-OPC (module 3) in WDM-PON to get optimal DCM in terms of dispersion compensation efficiency (DCE) and economical operation. As per author’s best knowledge, DCE calculations and performance enhancement with cost reduction using hybrid DCMs in ultra dense WDM-PON, is not reported so far. WDM-PON consists of 32 channels at 25 GHz channel spacing is analyzed for 300 km link distance at 10 Gbps/channel using different hybrid DCMs. It is perceived that highest DCE of 70% is given by module 3 with maximum cost reduction of 19.84%. DCE performance of three modules is as follows: Module 3 (DCE 70%), Module 1 (DCE 55%), Module 2 (DCE 45%) and cost reduction/increase from conventional module by 19.84% reduction (Module 3), 19.05% reduction (Module 1), and increase 10.5% (Module 2). Hence, Module 3 is preferred for long reach WDM-PON to get high performance with lesser cost.