Certain momentum correlations in the production and subsequent decay of heavy fermion pairs in $e^+e^-$ collisions have increased sensitivity to CP-violating electric and ``weak" dipole form factors of the fermion in the presence of longitudinal polarization of the $e^+e^-$ beams. However, unless the polarizations of $e^+$ and $e^-$ are equal and opposite, collinear initial-state radiation accompanied by helicity flip results in a CP-invariant contribution to these and acts as a background, and must be calculated and subtracted in order to isolate the CP-violating contribution due to the dipole form factor.We calculate this background contribution to such correlations, to order $\alpha$, in $e^+e^-\rightarrow \tau^+\tau^-$ and $e^+e^-\rightarrow t\bar{t}$. For the anticitipated luminosities at $\tau$-charm factories and SLC, for the $\tau$, and at a future linear collider with centre-of-mass energy of 500 GeV for the top, these backgrounds are smaller than the statistical background and may be safely neglected.