We consider frustrated Heisenberg antiferromagnets, whose clean-limit ground state is characterized by non-collinear long-range order with non-zero vector chirality, and study the effects of quenched bond disorder, i.e., random exchange couplings. A single bond defect is known to induce a dipolar texture in the spin background independent of microscopic details. Using general analytical arguments as well as large-scale simulations for the classical triangular-lattice Heisenberg model, we show that any finite concentration of such defects destroys long-range order for spatial dimension $d\leq 2$, in favor of a glassy state whose correlation length in $d=2$ is exponentially large for small randomness. Our results are relevant for a wide range of layered frustrated magnets.