We construct an inspiral-merger-ringdown eccentric gravitational-wave (GW) model for binary black holes with non-precessing spins within the effective-one-body formalism. This waveform model, SEOBNRv4EHM, extends the accurate quasi-circular SEOBNRv4HM model to eccentric binaries by including recently computed eccentric corrections up to 2PN order in the gravitational waveform modes, notably the $(l,|m|)=(2,2),(2,1),(3,3),(4,4),(5,5)$ multipoles. The waveform model reproduces the zero eccentricity limit with an accuracy comparable to the underlying quasi-circular model, with the unfaithfulness of $\lesssim1\%$ against quasi-circular numerical-relativity (NR) simulations. When compared against 28 public eccentric NR simulations from the Simulating eXtreme Spacetimes catalog with initial orbital eccentricities up to $e\simeq0.3$ and dimensionless spin magnitudes up to $+0.7$, the model provides unfaithfulness $<1\%$, showing that both the $(2,|2|)$-modes and the higher-order modes are reliably described without calibration to NR datasets in the eccentric sector. The waveform model SEOBNRv4EHM is able to qualitatively reproduce the phenomenology of dynamical captures, and can be extended to include spin-precession effects. It can be employed for upcoming observing runs with the LIGO-Virgo-KAGRA detectors and used to re-analyze existing GW catalogs to infer the eccentricity parameters for binaries with $e\lesssim0.3$ (at 20 Hz or lower) and spins up to $\lesssim 0.9-0.95$. The latter is a promising region of the parameter space where some astrophysical formation scenarios of binaries predict mild eccentricity in the ground-based detectors' bandwidth. Assessing the accuracy and robustness of the eccentric waveform model SEOBNRv4EHM for larger eccentricities and spins will require comparisons with, and, likely, calibration to eccentric NR waveforms in a larger region of the parameter space.