This work introduces novel results on the stability of linear time-varying discrete-time delay systems, addressing both internal stability and input-to-state stability. We start from the special case of positive delay systems and provide conditions of global exponential stability, both delay-dependent and delay-independent. Moreover, we formulate an explicit bound on the exponential decay rate of the state evolution, as a function of the maximum delay. We then show that the aforementioned stability conditions also prove input-to-state stability. The stability conditions consist of simple linear inequalities, albeit infinite-many due to the time-varying nature of the system. To overcome the computational burden, we also introduce a simple relaxation that allows to check the stability with a finite test, at the expense of conservatism. Finally, leveraging comparison systems and results from the theory of non-negative matrices, we generalize all the aforementioned results to systems with no sign constraint, for which we still provide stability conditions by means of linear inequalities and guaranteed convergence rate. Some consequences and a comparative example are provided for the well investigated special case of systems with constant matrices and time-varying delays.