Human immunodeficiency virus type 1 (HIV-1)-neutralizing antibodies (nAbs) that prevent infection are the main goal of HIV vaccine discovery. But as no nAb-eliciting vaccines are yet available, only data from HIV-1 neutralizers—persons with HIV-1 who naturally develop broad and potent nAbs—can inform about the dynamics and durability of nAb responses in humans, knowledge which is crucial for the design of future HIV-1 vaccine regimens. To address this, we assessed HIV-1-neutralizing immunoglobulin G (IgG) from 2,354 persons with HIV-1 on or off antiretroviral therapy (ART). Infection with non-clade B viruses, CD4+ T cell counts <200 µl−1, being off ART and a longer time off ART were independent predictors of a more potent and broad neutralization. In longitudinal analyses, we found nAb half-lives of 9.3 and 16.9 years in individuals with no- or low-level viremia, respectively, and 4.0 years in persons who newly initiated ART. Finally, in a potent HIV-1 neutralizer, we identified lower fractions of serum nAbs and of nAb-encoding memory B cells after ART initiation, suggesting that a decreasing neutralizing serum activity after antigen withdrawal is due to lower levels of nAbs. These results collectively show that HIV-1-neutralizing responses can persist for several years, even at low antigen levels, suggesting that an HIV-1 vaccine may elicit a durable nAb response.