Irradiation with 2.5 MeV electrons at doses up to 5.2 $\times$10$^{19}$ electrons/cm$^2$ was used to introduce point-like defects in single crystals of Ba$_{1-x}$K$_x$Fe$_2$As$_2$ with $x=$ 0.19 ($T_c=$ 14 K), $x=$ 0.26 ($T_c=$ 32 K) and 0.34 ($T_c=$ 39 K) to study the superconducting gap structure by probing the effect of non-magnetic scattering on electrical resistivity, $\rho(T)$, and London penetration depth, $ \lambda (T)$. For all compositions, the irradiation suppressed the superconducting transition temperature, $T_c$ and increased resistivity. The low - temperature behavior of $\lambda (T)$ is best described by the power - law function, $\Delta \lambda (T) = A(T/T_c)^n$. While substantial suppression of $T_c$ supports $s_{\pm}$ pairing mechanism, in samples close to the optimal doping, $x=$ 0.26 and 0.34, the exponent $n$ remained high ($n \geq 3$) indicating robust full superconducting gaps. For the $x=$ 0.19 composition, exhibiting coexistence of superconductivity and long - range magnetism, the suppression of $T_c$ was much more rapid and the exponent $n$ decreased toward dirty limit of $n$ = 2. In this sample, the irradiation also suppressed the temperature of structural/magnetic transition, $T_{sm}$, from 103 K to 98 K consistent with the itinerant nature of the magnetic order. Our results suggest that underdoped compositions, especially in the coexisting regime are most susceptible to non-magnetic scattering and imply that in multi-band Ba$_{1-x}$K$_x$Fe$_2$As$_2$ superconductors, the ratio of the inter-band to intra-band pairing strength, and associated gap anisotropy, increases upon the departure from the optimal doping.