We extend the method used in an earlier paper by the present authors to calculate here the effect of surface roughness on the van der Waals force between two different, semi-infinite dielectric media, separated by a region of vacuum of nominal thickness $l$, when both surfaces of the two media are rough. The result obtained has the form $f(l)=\ensuremath{-}\frac{{c}_{3}}{{(\frac{l}{a})}^{3}}\ensuremath{-}(\frac{{\ensuremath{\delta}}^{2}}{{a}^{2}})[\frac{{c}_{4}}{{(\frac{l}{a})}^{4}}+\frac{{c}_{5}}{{(\frac{l}{a})}^{5}}+\ensuremath{\cdots}]+O(\frac{{\ensuremath{\delta}}^{4}}{{a}^{4}})$ in the limit $\frac{l}{a}$ is large. Here $a$ is the transverse correlation length, the mean distance between consecutive peaks and valleys on the rough surface, while $\ensuremath{\delta}$ is the root-mean-square departure of the surface from flatness. For simplicity, $a$ and $\ensuremath{\delta}$ are assumed to be the same for both surfaces. The terms in the series multiplying the factor ($\frac{{\ensuremath{\delta}}^{2}}{{a}^{2}}$) have been calculated through terms of $O({(\frac{l}{a})}^{\ensuremath{-}7})$ for three different assumptions about the correlation between the roughness profile functions on the two surfaces. In addition, in the Appendix we show that a simple effective medium model of surface roughness, that has previously been shown to reproduce accurately the effects of surface roughness on the image potential and on the surface-plasmon dispersion curve, also yields the leading term in the roughness-induced contribution to the van der Waals force for large ($\frac{l}{a}$) with semiquantitative accuracy.