We study a recently introduced and exactly solvable mean-field model for the density of vibrational states $\mathcal{D}(\ensuremath{\omega})$ of a structurally disordered system. The model is formulated as a collection of disordered anharmonic oscillators, with random stiffness $\ensuremath{\kappa}$ drawn from a distribution $p(\ensuremath{\kappa})$, subjected to a constant field $h$ and interacting bilinearly with a coupling of strength $J$. We investigate the vibrational properties of its ground state at zero temperature. When $p(\ensuremath{\kappa})$ is gapped, the emergent $\mathcal{D}(\ensuremath{\omega})$ is also gapped, for small $J$. Upon increasing $J$, the gap vanishes on a critical line in the $(h,J)$ phase diagram, whereupon replica symmetry is broken. At small $h$, the form of this pseudogap is quadratic, $\mathcal{D}(\ensuremath{\omega})\ensuremath{\sim}{\ensuremath{\omega}}^{2}$, and its modes are delocalized, as expected from previously investigated mean-field spin glass models. However, we determine that for large enough $h$, a quartic pseudogap $\mathcal{D}(\ensuremath{\omega})\ensuremath{\sim}{\ensuremath{\omega}}^{4}$, populated by localized modes, emerges, the two regimes being separated by a special point on the critical line. We thus uncover that mean-field disordered systems can generically display both a quadratic-delocalized and a quartic-localized spectrum at the glass transition.