With the rapid development of the fifth-generation (5G) mobile communication technology, the application of each frequency band has reached the extreme, causing mutual interference between different modules. Hence, there is a requirement for detecting filtering and preventing interference. In the troposphere, over-the-horizon propagation occurs in atmospheric ducts and turbulent media. The effects of both ducting and turbulence can increase the probability of occurrence of long-distance co-channel interference (CCI), in turn, severely affecting the key performance indicators such as system access, handover and drop. In the 5G era, to ensure communication channels and information security, CCI must be reduced. This paper introduces a scattering parabolic equation algorithm for calculating signal propagation in atmospheric ducts on irregular terrain boundaries. It combines Hitney’s radio physical optical model and Wagner’s nonuniform turbulent scattering model for calculating the tropospheric scattering in an evaporation duct or a surface-based duct. The new model proposes a tropospheric scattering parabolic equation algorithm for various tropospheric duct environments. Finally, as a specific case, the topographical boundaries between several cities in the East China Plain were considered, and the over-the-horizon propagation loss was simulated for various ducting and turbulent environments. The simulation results were used to evaluate whether CCI would occur between cities in a specific environment.