The secretory glutaminyl cyclase (sQC) and Golgi-resident glutaminyl cyclase (gQC) are responsible for N-terminal protein pyroglutamation and associated with various human diseases. Although several sQC/gQC inhibitors have been reported, only one inhibitor, PQ912, is currently undergoing clinic trials for the treatment of Alzheimer's disease. We report an X-ray crystal structure of sQC complexed with PQ912, revealing that the benzimidazole makes "anchor" interactions with the active site zinc ion and catalytic triad. Structure-guided design and optimization led to a series of new benzimidazole derivatives exhibiting nanomolar inhibition for both sQC and gQC. In a MPTP-induced Parkinson's disease (PD) mouse model, BI-43 manifested efficacy in mitigating locomotor deficits through reversing dopaminergic neuronal loss, reducing microglia, and decreasing levels of the sQC/gQC substrates, α-synuclein, and CCL2. This study not only offers structural basis and new leads for drug discovery targeting sQC/gQC but also provides evidence supporting sQC/gQC as potential targets for PD treatment.