We present a relativistic self-consistent theory of the coherent stopping power (CSP) of ultradense charged particle beams propagating in a dense matter. CSP corresponds to a collective inelastic collision which adds to the ordinary stopping power of individual particles. Unlike the latter, which depends only on the particles' energy, CSP depends upon many more parameters such as the total charge of the ensemble and its charge density and shape. This paves the way for a broad variety of novel methods to tailor particle absorption and penetration in a dense matter. CSP losses can be explained both by collective excitations of single or multiple molecules and by the emission of coherent Cherenkov radiation. Here we demonstrate that the former mechanism is more relevant than the latter. We find that the coherent energy absorption of subpicosecond particle bunches in water occurs exciting a broad Debye process in the GHz range and an intermolecular stretching vibration mode in the THz region. We generalize the Bethe-Bloch stopping power formula to coherent effects including self-forces, dielectric screening, and absorption by the dense matter. For the sake of a self-consistent dynamical theory including phase space evolution, we have also generalized the Fermi-Eyges theory of particle diffusion to the presence of forces. Nonlinear dynamics is demonstrated, inducing nonlinear dose release, beam self-focusing, and self-enhancement of coherent losses. Given the advent of ultraintense particle sources and their use for biomedical and other relevant applications, our results may be of paramount importance for contemporary and future developments of science and technology. Published by the American Physical Society 2024