In this work, we study the spectral properties of the adjacency matrix of critical Erd\"os-R\'enyi (ER) graphs, i.e., when the average degree is of order $lnN$. In a series of recent inspiring papers, Alt, Ducatez, and Knowles have rigorously shown that these systems exhibit a ``semilocalized'' phase in the tails of the spectrum where the eigenvectors are exponentially localized on a subextensive set of nodes with anomalously large degree. We propose two approximate analytical strategies to analyze this regime based respectively on the simple ``rules of thumb'' for localization and ergodicity and on an approximate treatment of the self-consistent cavity equation for the resolvent. Both approaches suggest the existence of two different regimes: a fully Anderson localized phase at the spectral edges, in which the eigenvectors are localized around a unique center, and an intermediate partially delocalized but nonergodic phase, where the eigenvectors spread over many resonant localization centers. In this phase, the exponential decay of the effective tunneling amplitudes between the localization centers is counterbalanced by the large number of nodes towards which tunneling can occur, and the system exhibits minibands in the local spectrum over which the Wigner-Dyson statistics establishes. We complement these results by a detailed numerical study of the finite-size scaling behavior of several observables that supports the theoretical predictions and allows us to determine the critical properties of the two transitions. Critical ER graphs provide a pictorial representation of the Hilbert space of a generic many-body Hamiltonian with short-range interactions. In particular, we argue that their phase diagram can be mapped onto the out-of-equilibrium phase diagram of the quantum random energy model.