Compared with traditional duplex technology, in-band full-duplex (IBFD) technology can improve spectrum utilization, network capacity and system throughput. However, strong co-frequency signals from adjacent transmission antennas may cause the received signals to be flooded, significantly degrading the communication performance. In addition, frequency down-conversion is an essential function of the receiver, and the subsequent image interference will also block the receiver. A self-interference cancellation (SIC) and image rejection down-conversion receiver based on the Sagnac loop is presented to solve these two problems. The self-interfering signals are eliminated using polarization control and delay matching in the optical domain. Image rejection is achieved by adjusting the DC bias of the modulator. Therefore, all operations required for the above two functions are implemented using optical methods. The theoretical and experimental validation is carried out. The results show that a self-interference cancellation depth of over 43 dB and an image interference rejection ratio of 45 dB is achieved for single-tone signal after frequency down-conversion. For 16 quadrature amplitude modulation (16QAM) signals with a bandwidth range from 20 MHz to 50 MHz, the self-interference cancellation depth exceeds 26 dB, and the image interference rejection ratio exceeds 30 dB.