In this paper, we demonstrate for the first time how the Integrated Finite Element Neural Network (I-FENN) framework, previously proposed by the authors [1,2], can efficiently simulate the entire loading history of non-local gradient damage propagation. To achieve this goal, we first adopt a Temporal Convolutional Network (TCN) as the neural network of choice to capture the history-dependent evolution of the non-local strain in a coarsely meshed domain. The quality of the network predictions governs the computational performance of I-FENN, and therefore we perform an extended investigation aimed at enhancing them. We explore a data-driven vs. physics-informed TCN setup to arrive at an optimum network training, evaluating the network based on a coherent set of relevant performance metrics. We address the crucial issue of training a physics-informed network with input data that span vastly different length scales by proposing a systematic way of input normalization and output un-normalization. We then integrate the trained TCN within the nonlinear iterative FEM solver and apply I-FENN to simulate the damage propagation analysis. I-FENN is always applied in mesh idealizations different from the one used for the TCN training, showcasing the framework’s ability to be used at progressively refined mesh resolutions. We illustrate several cases that I-FENN completes the simulation using either a modified or a full Newton–Raphson scheme, and we showcase its computational savings compared to both the classical monolithic and staggered FEM solvers. We underline that we satisfy very strict convergence criteria for every increment across the entire simulation, providing clear evidence of the robustness and accuracy of I-FENN. All the code and data used in this work will be made publicly available upon publication of the article.