In this paper, we present, develop and investigate a special kind of recurrent neural network termed Zhang neural network (ZNN) for time-varying matrix inversion. Comparing with the dynamic system proposed by Getz and Marsden (G-M) for time-varying matrix inversion, we show that such a G-M dynamic system depicted in an explicit dynamics can also be derived from the presented ZNN model depicted in an implicit dynamics. In other words, a novel result on the relationship between the ZNN model and others' model/method (i.e., the G-M dynamic system) is found for time-varying matrix inversion. In addition, we propose and investigate the discrete-time algorithms (depicted by systems of difference equations) of the aforementioned ZNN and G-M models in two situations, i.e., the time-derivative of the time-varying matrix to be inverted being known or unknown. Simulative and numerical results demonstrate the superior performance of the ZNN models for time-varying matrix inversion, as well as the efficacy of the G-M dynamic system (which has to be started with initial conditions sufficiently close to the desired initial inverse). Furthermore, the ZNN models and G-M dynamic system are applied to the kinematic control of a two-link planar manipulator via online solution of time-varying matrix inversion.