Mode converter (MC) is a crucial device in optical information processing based on mode division multiplexing. Recently, MCs based on metasurface have attracted much attention due to their compact footprints. However, this type of MCs is usually achieved by introducing refractive index modulation in the transverse direction. As a result, many interfaces are unavoidable, which is the primary origin of high loss and narrow bandwidth. In this paper, we propose a new mechanism of ultra-compact, high efficiency, and broadband MC based on longitudinal refractive index modulation. The proposed dielectric modulation only introduces a few longitudinal pixels in the waveguide, resulting in weak scattering and weak resonance refractive index modulation with low crosstalk, and large bandwidth. By adopting the inverse design algorithm, the specified TE0-TE1 and TE0-TE2 mode conversions are obtained with superior performances including the ultra-compact footprint, high conversion efficiency, and low crosstalk over a wide bandwidth. This method could be extended to other desired mode conversions and also has excellent potential in optical integrated circuits and optical information processing.