We have investigated one-way surface plasmon-polaritons (SPPs) at the interface of a continuum magnetoplasma material and metal, in the presence of three-dimensional surface defects. Bulk electromagnetic modes of continuum materials have Chern numbers, analogous to those of photonic crystals. This can lead to the appearance of topologically-protected surface modes at material interfaces, propagating at frequencies inside the bandgap of the bulk materials. Previous studies considered two-dimensional structures; here we consider the effect of three-dimensional defects, and show that, although backward propagation/reflection cannot occur, side scattering does take place and has significant effect on the propagation of the surface mode. Several different waveguiding geometries are considered for reducing the effects of side-scattering, and we also consider the effects of metal loss.