An optically enabled multipath self-interference (MSI) cancellation technique for in-band full duplex (IBFD) multiple input multiple output (MIMO) Radio over Fiber RoF systems under Rayleigh fading channels is proposed. A digitally assisted Rayleigh fading channel model generates the MSI signal. An optically tunable multipath delay lines (OTMDL) module is designed using optical delay lines and an optical attenuator for coarse and fine-tuning of time delays and amplitude matching between the locally generated reference (LR) signal and the MSI signal. The OTMDL module adjusts the LR signal to cancel the effect of MSI on the signal received through the fading channel. The concept is validated using a remote sensor node of a 3 × 3 IBFD MIMO RoF system with a Rayleigh fading channel. The system's performance is evaluated by determining the MSI cancellation depths using the OTMDL section for various cancellation channels and by computing the MSI cancellation for both single band and wideband signals across different cancellation channels. An average value of MSI cancellation depth of 20.38 dB is obtained for a single-tone radio frequency signal in the range 15–25 GHz. Thus, the proposed system for MSI cancellation can be utilized in wireless communication systems in multiple ways.