Open systems with anti-parity-time ({{{{{{{mathcal{APT}}}}}}}}) or {{{{{{{mathcal{PT}}}}}}}} symmetry exhibit a rich phenomenology absent in their Hermitian counterparts. To date all model systems and their diverse realizations across classical and quantum platforms have been local in time, i.e., Markovian. Here we propose a non-Markovian system with anti-{{{{{{{mathcal{PT}}}}}}}}-symmetry where a single time-delay encodes the retention of memory, and experimentally demonstrate its consequences with two time-delay coupled semiconductor lasers. A transcendental characteristic equation with infinitely many eigenvalue pairs sets our model apart. We show that a sequence of amplifying-to-decaying dominant mode transitions is induced by the time delay in our minimal model. The signatures of these transitions quantitatively match results obtained from four, coupled, nonlinear rate equations for laser dynamics, and are experimentally observed as constant-width sideband oscillations in the laser intensity profiles. Our work introduces a paradigmatic non-Hermitian system with memory, paves the way for its realization in classical systems, and may apply to time-delayed feedback-control for quantum systems.