Maxwell's equations in a bounded Debye medium are formulated in terms of the standard partial differential equations of electromagnetism with a Volterra-type history dependence of the polarization on the electric field intensity. This leads to Maxwell's equations with memory. We make a correspondence between this type of constitutive law and the hereditary integral constitutive laws from linear viscoelasticity, and we are then able to apply known results from viscoelasticity theory to this Maxwell system. In particular, we can show long-time stability by shunning Gronwall's lemma and estimating the history kernels more carefully by appeal to the underlying physical fading memory. We also give a fully discrete scheme for the electric field wave equation and derive stability bounds which are exactly analogous to those for the continuous problem, thus providing a foundation for long-time numerical integration. We finish by also providing error bounds for which the constant grows, at worst, linearly in time (excluding the time dependence in the norms of the exact solution). Although the first (mixed) finite element error analysis for the Debye problem was given by Li (2007), this seems to be the first time sharp constants have been given for this problem.