We generalise the theory of mean-field galactic dynamos by allowing for temporal non-locality in the mean electromotive force (emf). This arises in random flows due to a finite response time of the mean emf to changes in the mean magnetic field, and leads to the telegraph equation for the mean field. The resulting dynamo model also includes the nonlinear dynamo effects arising from magnetic helicity balance. Within this framework, coherent large-scale magnetic spiral arms superimposed on the dominant axially symmetric magnetic structure are considered. A non-axisymmetric forcing of the mean-field dynamo by a spiral pattern (either stationary or transient) is invoked, with the aim of explaining the phenomenon of magnetic arms. For a stationary dynamo forcing by a rigidly rotating material spiral, we find corotating non-axisymmetric magnetic modes enslaved to the axisymmetric modes and strongly peaked around the corotation radius. For a forcing by transient material arms wound up by the galactic differential rotation, the magnetic spiral is able to adjust to the winding so that it resembles the material spiral at all times. There are profound effects associated with the temporal non-locality, i.e. finite `dynamo relaxation time'. For the case of a rigidly rotating spiral, a finite relaxation time causes each magnetic arm to mostly lag the corresponding material arm with respect to the rotation. For a transient material spiral that winds up, the finite dynamo relaxation time leads to a large, negative (in the sense of the rotation) phase shift between the magnetic and material arms, similar to that observed in NGC 6946 and other galaxies. We confirm that sufficiently strong random seed fields can lead to global reversals of the regular field along the radius whose long-term survival depends on specific features of a given galaxy.