We present a detailed analysis and comparison of dielectric waveguides made of CdTe, GaP, GaAs and InP for modal phase matched optical difference frequency generation (DFG) in the terahertz domain. From the form of the DFG equations, we derived the definition of a very general figure of merit (FOM). In turn, this FOM enabled us to compare different configurations, by taking into account linear and nonlinear susceptibility dispersion, terahertz absorption, and a rigorous evaluation of the waveguide modes properties. The most efficient waveguides found with this procedure are predicted to approach the quantum efficiency limit with input optical power in the order of kWs.