The utility of covalent crosslinkers as permanent netpoints has enabled robust shape memory behaviour in shape memory thermosets (SMTs). Having this permanent crosslinked structure, alternatively, challenges the reprogramming of permanent shapes. The introduction of vitrimeric type dynamic chemistry into SMTs allows for reconfiguration of crosslinked network and thus reprogramming of permanent shapes. However, adding this adaptability/plasticity to the network could potentially affect the shape memory behaviour, especially when the shape programming is conducted at a temperature that activates the network plasticity as well. Herein, to address the potential influence of network plasticity, we studied the shape memory behaviour of a shape memory vitrimer (SMV) that possesses a low network activation energy (Ea) of 33 kJ/mol, which indicates a relatively high plasticity index during shape programming. As we found, there is a dependence of shape recovery on the shape-programming time: a longer programming time leads to a poorer shape recovery. This negative correlation results from the dissipation of stored elastic energy at local strained regions associated with shape programming. We further related the loss of elastic energy to the stress-relaxation that results from chain exchanges. Accordingly, the dependence can be analyzed with an analog of stress-relaxation model. We reasoned that this latent plasticity associated with dynamic chain exchanges is ubiquitous in SMVs, and so one needs to carefully evaluate the reprogramming temperature and time when performing shape memory studies on SMVs.