The Langevin equation for a particle ("random walker") moving in d-dimensional space under an attractive central force and driven by a Gaussian white noise is considered for the case of a power-law force, F(r) approximately -r(-sigma). The "persistence probability," P0(t), that the particle has not visited the origin up to time t is calculated for a number of cases. For sigma>1, the force is asymptotically irrelevant (with respect to the noise), and the asymptotics of P0(t) are those of a free random walker. For sigma<1, the noise is (dangerously) irrelevant and the asymptotics of P0(t) can be extracted from a weak noise limit within a path-integral formalism employing the Onsager-Machlup functional. The case sigma=1, corresponding to a logarithmic potential, is most interesting because the noise is exactly marginal. In this case, P0(t) decays as a power law, P0(t) approximately t(-straight theta) with an exponent straight theta that depends continuously on the ratio of the strength of the potential to the strength of the noise. This case, with d=2, is relevant to the annihilation dynamics of a vortex-antivortex pair in the two-dimensional XY model. Although the noise is multiplicative in the latter case, the relevant Langevin equation can be transformed to the standard form discussed in the first part of the paper. The mean annihilation time for a pair initially separated by r is given by t(r) approximately r(2) ln(r/a) where a is a microscopic cutoff (the vortex core size). Implications for the nonequilibrium critical dynamics of the system are discussed and compared to numerical simulation results.