The Kalman filter is a linear optimal estimator when it can accurately receive measurements and the noise obeys a Gaussian distribution. However, the measurement noise may have heavy-tailed characteristics because of outlier interference. Actually, real-world measurement information may have one-step randomly delayed measurements (ORDMs) with unknown latency probability (LP) due to exogenous disturbances. Moreover, systems are usually not linear. To overcome the influence of the state estimator of nonlinear systems from heavy-tailed measurement noise (HMN) and the unknown LP, this paper focuses on the robust Gaussian approximate (GA) filter and smoother for the above issues. First, the one-step predicted probability density function (PDF) is modeled as a Gaussian distribution, and the likelihood PDF is modeled as a Normal–Gamma–Beta (NGBM) distribution. Furthermore, novel robust Gaussian approximate filter and smoother are proposed based on the variational Bayesian technique. They provide general estimation frameworks for estimating state of nonlinear systems with HMN and ORDMs, and different nonlinear estimators can be implemented using different numerical techniques. In addition, the computational complexity of the proposed algorithms is calculated. Finally, taking a univariate nonstationary growth model, a passive ranging problem, and target tracking as examples, the effectiveness and superiority of the proposed robust algorithms contrasted with the existing algorithms are shown. The outcomes show that the proposed methods have higher estimation precision; however, the computational burden is marginally increased.