The evolution of magnetic domain structures with temperature during magnetic reversal in $\mathrm{Co}(4\phantom{\rule{0.3em}{0ex}}\mathrm{\AA{}})∕\mathrm{Pt}({t}_{\mathrm{Pt}})$ multilayers with perpendicular anisotropy has been investigated using magneto-optical Kerr imaging. Relatively large Pt layer thicknesses (${t}_{\mathrm{Pt}}=43$ and $63\phantom{\rule{0.3em}{0ex}}\mathrm{\AA{}}$) are chosen for this study because the interlayer coupling strength in the multilayers varies from weak at room temperature to strong at low temperatures. A $\mathrm{Co}∕\mathrm{Pt}$ multilayer with strong interlayer coupling $({t}_{\mathrm{Pt}}=11\phantom{\rule{0.3em}{0ex}}\mathrm{\AA{}})$ is also studied as a comparison. Kerr imaging during magnetic reversal shows a transformation of domain patterns with temperature which correlates directly with the enhancement of interlayer exchange coupling with decreasing temperature, as well as the conversion from domain-wall-propagation-dominant reversal at room temperature to nucleation-dominant reversal at low temperatures. The enhanced interlayer coupling at low temperatures leads to the entire multilayer switching as a single ferromagnet; while at higher temperatures, when the interlayer coupling weakens, quasi-independent layer-by-layer magnetic reversal is observed. The transformation from propagation- to nucleation-dominant magnetic reversal can be understood by the competition between activation energies for domain nucleation and propagation, Zeeman energy, and thermal energy.