In this paper, a novel Taylor-type difference rule with O(τ4) pattern error is provided for the first-order derivative approximation. Then, a high accuracy noise-tolerant five-step discrete-time zeroing neural network (ZNN) (termed as FDNTZNN model) is proposed to solve the time-varying matrix inversion problem in real-time. In addition, to obtain the derivative value of time-varying variables in real-world applications, the backward-difference rule is exploited to develop the FD-NTZNN model when the derivative information is unknown (FD-NTZNN-U). Theoretical analysis demonstrates that the proposed FD-NTZNN models have the properties of 0−stability, consistency and convergence. For comparative analysis, the classical Euler-type discrete-time ZNN model (EDZNN), five-step Taylor-type discrete-time ZNN model (FDZNN) and Euler-type discrete-time noise-tolerant ZNN (NTZNN) model (ED-NTZNN) are reconsidered. Ultimately, two illustrative numerical simulations and an application example to motion generation of manipulator are simulated to substantiate the feasibility and effectiveness of the proposed FD-NTZNN model and FD-NTZNN-U model for online time-varying matrix inversion in the presence of different types of noise.