The search for the QCD critical point in heavy-ion collision experiments requires dynamical simulations of the bulk evolution of QCD matter as well as of fluctuations. We consider two essential ingredients of such a simulation: a generic extension of hydrodynamics by a_parametrically_ slow mode or modes ("Hydro+") and a description of fluctuations out of equilibrium. By combining the two ingredients we are able to describe the bulk evolution and the fluctutations within the same framework. Critical slowing down means that equilibration of fluctuations could be as slow as hydrodynamic evolution and thus fluctuations could significantly deviate from equilibrium near the critical point. We generalize hydrodynamics to partial-equilibrium conditions where the state of the system is characterized by the off-equilibrium magnitude of fluctuations in addition to the usual hydrodynamic variables -- conserved densities. We find that the key element of the new formalism -- the extended entropy taking into account the off-equilibrium fluctuations -- is remarkably similar to the 2PI action in quantum field theory. We show how the new Hydro+ formalism reproduces two major effects of critical fluctuations on the bulk evolution: the strong frequency dependence of the anomalously large bulk viscosity as well as the stiffening of the equation of state with increasing frequency or wave-number. While the agreement with known results confirms its validity, the fact that Hydro+ achieves this within a local and deterministic framework gives it significant advantages for dynamical simulations.