Electric-field switching of polarization is the building block of a wide variety of ferroelectric devices. In turn, understanding the factors affecting ferroelectric switching and developing routes to control it are of great technological significance. This work provides systematic experimental evidence of the role of defects in affecting ferroelectric-polarization switching and utilizes the ability to deterministically create and spatially locate point defects in $\mathrm{PbZ}{\mathrm{r}}_{0.2}\mathrm{T}{\mathrm{i}}_{0.8}{\mathrm{O}}_{3}$ thin films via focused-helium-ion bombardment and the subsequent defect-polarization coupling as a knob for on-demand control of ferroelectric switching (e.g., coercivity and imprint). At intermediate ion doses ($0.22--2.2\ifmmode\times\else\texttimes\fi{}{10}^{14}\phantom{\rule{0.16em}{0ex}}\mathrm{ions}\phantom{\rule{0.16em}{0ex}}{\mathrm{cm}}^{\ensuremath{-}2}$), the dominant defects (isolated point defects and small clusters) show a weak interaction with domain walls (pinning potentials from $200--500\phantom{\rule{0.16em}{0ex}}\mathrm{K}\phantom{\rule{0.16em}{0ex}}\mathrm{MV}\phantom{\rule{0.16em}{0ex}}{\mathrm{cm}}^{\ensuremath{-}1}$), resulting in small and symmetric changes in the coercive field. At high doses ($0.22--1\ifmmode\times\else\texttimes\fi{}{10}^{15}\phantom{\rule{0.16em}{0ex}}\mathrm{ions}\phantom{\rule{0.16em}{0ex}}{\mathrm{cm}}^{\ensuremath{-}2}$), on the other hand, the dominant defects (larger defect complexes and clusters) strongly pin domain-wall motion (pinning potentials from 500 to $1600\phantom{\rule{0.16em}{0ex}}\mathrm{K}\phantom{\rule{0.16em}{0ex}}\mathrm{MV}\phantom{\rule{0.16em}{0ex}}{\mathrm{cm}}^{\ensuremath{-}1}$), resulting in a large increase in the coercivity and imprint, and a reduction in the polarization. This local control of ferroelectric switching provides a route to produce novel functions; namely, tunable multiple polarization states, rewritable pre-determined 180\ifmmode^\circ\else\textdegree\fi{} domain patterns, and multiple zero-field piezoresponse and permittivity states. Such an approach opens up pathways to achieve multilevel data storage and logic, nonvolatile self-sensing shape-memory devices, and nonvolatile ferroelectric field-effect transistors.