Topological pumps have garnered substantial attention in physics. However, the requirement for slow evolution speed to satisfy adiabaticity greatly restricts their application in on-chip devices. Here, we discover a direct link between adiabaticity and quantum metric, the real part of quantum geometry that has been relatively less explored compared to its imaginary counterpart, the Berry curvature. We demonstrate that the evolution speed of topological pumps between nontrivial edge states can be increased by reducing the quantum metric via introduction of long-range coupling to the celebrated Rice-Mele model. This fast topological pump can occur without affecting the bulk state evolution, which challenges the common understanding. We experimentally confirm our findings by using a platform consisting of bilayer integrated silicon waveguides operating at telecommunication wavelengths. Our work provides possibilities for lifting topological pumps from the constraints of slow evolution and paves the way toward compact photonic integration.