Recent scanning tunnelling microscopy (STM) experiments on underdoped cuprates have displayed modulations in the local electronic density of states which are centered on a Cu-O-Cu bond (Kohsaka et. al., cond-mat/0703309). As a paradigm of the pinning of such bond-centered ordering in strongly correlated systems, we present the theory of valence bond solid (VBS) correlations near a single impurity in a square lattice antiferromagnet. The antiferromagnet is assumed to be in the vicinity of a quantum transition from a magnetically ordered Neel state to a spin-gap state with long-range VBS order. We identify two distinct classes of impurities: i) local modulation in the exchange constants, and ii) a missing or additional spin, for which the impurity perturbation is represented by an uncompensated Berry phase. The `boundary' critical theory for these classes is developed: in the second class we find a `VBS pinwheel' around the impurity, accompanied by a suppression in the VBS susceptibility. Implications for numerical studies of quantum antiferromagnets and for STM experiments on the cuprates are noted.