For state-of-charge (SOC) estimation with high precision, moving horizon estimation (MHE) has recently emerged as a competitive alternative to various Kalman filters or observers due to its significantly improved accuracy and robustness. However, MHE demands considerably higher computational complexity, preventing its deployment on a low-cost embedded microcontroller. To address this challenge, we propose a fast moving horizon SOC estimation algorithm to retain the benefits of MHE at a vastly reduced computational cost. In particular, we consider joint SOC and parameter estimation for an SOC-dependent equivalent circuit model subject to model mismatch. The proposed fast joint MHE (jMHE) algorithm performs a fixed number of Gauss–Newton (GN) iterations to approximate the fully converged solution. At each iteration, the GN Hessian matrix is factorized by exploiting its block tridiagonal structure to construct computationally efficient forward–backward recursions. To further speed up computations, another fast jMHE algorithm wtih an Event-Triggered Relinearization strategy (jMHE-ETR) is proposed to avoid refactorizing the GN Hessian matrix at each iteration. Using experimental datasets under different operating conditions, it is verified on both laptop and Raspberry Pi microcontroller that compared to the conventional optimal MHE solved with an off-the-shelf solver, the proposed fast jMHE and jMHE-ETR algorithms both have a slight loss of performance (still significantly better than extended Kalman filtering) while reducing its computational cost by an order of magnitude.