Terahertz communications has emerged as an excellent candidate for the next generation of wireless communication networks. Despite of significant advances, one still looks for a high-efficiency terahertz source using nonlinear optics, such as optical refraction and difference-frequency generation. Concurrently, recent years have witnessed significant progress in metamaterials-assisted nonlinear optical processes. In this study, we propose a theoretical approach using near zero-index nonlinear hybrid waveguides to realize a high generation efficiency of terahertz waves via on-chip difference-frequency generation. Remarkably, thanks to the simultaneous realization of near zero-index both at terahertz and telecom frequencies, such a nonlinear hybrid waveguide can overcome two major bottlenecks in the conversion from optical wave to terahertz waves: phase mismatch and weak nonlinear optical-terahertz waves interactions, resulting in a high conversion as 2.2 × 10−3W−1 just within the propagation length of several terahertz wavelengths. Our results demonstrate the feasibility of an on-chip terahertz source, offering unique properties such as high integration and ultra-low power consumption for terahertz communication systems.