High-speed photonic networks using digital signal processing (DSP) techniques are flourishing nowadays to meet the high-bandwidth requirements of modern bandwidth-thirsty applications in a cost-effective manner. However, the additional latency introduced by DSP is hindering the latency-critical applications. In this paper, a FPGA-based real-time low-latency four-level pulse amplitude modulation (PAM-4) receiver including digital adaptive equalization (DAE) is designed and implemented by using a latency-reducing parallel architecture. The DSP-introduced latency in the receiver end is analyzed in detail. As for DAE parallel implementation, a novel re-allocation scheme is proposed to cope with the issue of the dependency of the output on the successive input samples, and a look-ahead computation technique is introduced to improve the adaptive update efficiency. A real-time PAM-4 receiver is demonstrated in an experimental fiber link with 2.5 Gbit/s data rate for the performance evaluation. Compared with offline processing with MATLAB, the BER performance has little deterioration at 7% FEC limit of 1 × 10−3. With the help of the proposed deep-parallel technique, the DSP-introduced latency is reduced to 0.4μs on average, which better meets the requirements of latency-sensitive user cases in 5G networks. Furthermore, the real-time PAM-4 receiver could be flexibly reconfigured for various scenarios with low-latency requirements, and the latency-efficient parallel technique as well as the latency analysis method can also be extended to high-speed hardware implementation for data rates up to 100 Gbit/s or more.