Accurate assessment of the damage to a structure is the key to structural health monitoring. Among them, one effective means to assess the real-time status of the structure is by establishing a finite element model (FEM) of the structure and updating it synchronously. Previous research mainly updates the FEM of the structure with a certain moment in time without considering the continuous changes of structural parameters over a longer time. In this paper, we propose a FEM updating method for steel truss structure based on knowledge-enhanced deep reinforcement learning, which introduces long short-term memory (LSTM) network based on a deep deterministic policy gradient (DDPG) algorithm and constructs a knowledge-enhanced noise function (NF) module based on parametric analysis. The NF-LSTM-DDPG algorithm can fit a continuous variation of structural parameters. The ablation experiments revealed that the LSTM network and the NF module facilitate the exploration and learning ability of the DDPG algorithm. The NF-LSTM-DDPG algorithm is used in an actual steel truss structure case, and the variation rules of the elastic modulus and support deviations obtained from the solution are consistent with the actual situation. The average relative error of the 34 strain measurement points after updating is 14.147 %, and the relative error is higher than 20 % in only 13 days out of 187 days of continuous monitoring. Five days were selected for the comparative study, and the experimental results show that the relative error of the NF-LSTM-DDPG algorithm is 13.206 %, which is higher than that of the particle swarm optimization algorithm and the simulated annealing algorithm. The change of elastic modulus updated based on the NF-LSTM-DDPG algorithm is more reasonable than other algorithms.