We are guided by the fact that zeroing neural networks (ZNN) are proven tool in online solving the time-varying (TV) matrix Moore–Penrose (M–P) inverse. This paper focuses on online computing TV full-row rank or full-column rank matrix M–P inverse using a novel ZNN model with an optimized activation function (AF) and improved error function (Zhangian). ZNN dynamical systems accelerated by the optimized class of AFs converge in a finite-time to the TV theoretical M–P inverse. The upper bounds of the estimated convergence time are obtained analytically using the Lyapunov stability theory. The simulation experiments support the theoretical analysis and demonstrate the effectiveness of the proposed ZNN dynamics.